Protocol buffers (that's what protobuf means :)) are a means to serialize structured data. Now you may be wondering what this 'serialize' means.
To be brief : Serialization is a process of packing the data structures or objects into sequence of bits so that they can be stored in a file or a buffer and transmitted across the network and then can be retrieved on the same or other system environment. Examples include XML, JSON, ASN.1, YAML, ProtoBuf etc.
Advantages :
1. Persistent object creation
2. RPC enabling
3. Are simple
4. Save space on disk
5. Very fast
6. Platform and language independent
ProtoBuf is available freely from Google (
http://code.google.com/p/protobuf/). Procedure of installation and compilation is present there.
Everything that needs to be serialized is put in a protocol buffer message (a .proto file). This file tells the compiler how the data is to be serialized.
For each field there are setter and getter functions fieldname() is a getter and set_fieldname() is a setter. Remember to use these functions as the exact way it is told here.
The tutorials for the different languages can be found
here
I present a small simple example to use the protobuf library and there is much more than that one can do with the protobuf :)
To make and run the example (on linux) follow the steps :
1. Download the tar.gz package from
here
2. Login as root
3. Untar it in your home directory (say user1)
4. Go inside the protobuf directory and run ./configure
5. Add the path /usr/local/lib to the environment variable LD_LIBRARY_PATH
6. Then run :
make
make install
7. You are now ready to use the library.
8. To use protobuf you will need following files (I am explaining for c++ only):
.cc file
.proto file
Makefile
Rest of the files are generated on the fly.
Here is an example of a Vehicle parts which contains three fields , two mandatory and one optional.
-Part ID (mandatory)
-Part Name (mandatory)
-Vendor (optional)
We write this in .proto file
//partbook.proto
package tutorial;
message Part {
required string name = 1;
required int32 partid = 2; // Unique ID number for this part
optional string vendor = 3;
}
message PartDetails {
repeated Part part = 1;
}
Then we write a c++ program to use the library and .proto to add serialized data to the data file.
/*add_part.cc*/
#include <iostream>
#include <fstream>
#include <string>
#include "partbook.pb.h"
using namespace std;
/* This function fills in a Part message based on user input.*/
void PromptForPart(tutorial::Part* part) {
cout << "Enter part ID number: ";
int partid;
cin >> partid;
part->set_partid(partid);
cin.ignore(256, '\n');
cout << "Enter part name: ";
getline(cin, *part->mutable_name());
cout << "Enter vendor name (blank for none): ";
string vendor;
getline(cin, vendor);
if (!vendor.empty()) {
part->set_vendor(vendor);
}
}
/* Main function: Reads the entire part list from a file,
* adds one part based on user input, then writes it back out to the same
* file.
*/
int main(int argc, char* argv[]) {
/* Verify that the version of the library that we linked against is
* compatible with the version of the headers we compiled against.
*/
GOOGLE_PROTOBUF_VERIFY_VERSION;
if (argc != 2) {
cerr << "Usage: " << argv[0] << " PART_BOOK_FILE" << endl;
return -1;
}
tutorial::PartDetails part_book;
{
// Read the existing part book.
fstream input(argv[1], ios::in | ios::binary);
if (!input) {
cout << argv[1] << ": File not found. Creating a new file." << endl;
} else if (!part_book.ParseFromIstream(&input)) {
cerr << "Failed to parse part book." << endl;
return -1;
}
}
// Add a new part.
PromptForPart(part_book.add_part());
{
// Write the new part book back to disk.
fstream output(argv[1], ios::out | ios::trunc | ios::binary);
if (!part_book.SerializeToOstream(&output)) {
cerr << "Failed to write part book." << endl;
return -1;
}
}
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
Then we write a c++ program to use the library and .proto to serialize the data.
/*list_parts.cc*/
#include <iostream>
#include <fstream>
#include <string>
#include "partbook.pb.h"
using namespace std;
// Iterates though all parts in the PartDetails and prints info about them.
void ListParts(const tutorial::PartDetails& part_book) {
for (int i = 0; i < part_book.part_size(); i++) {
const tutorial::Part& part = part_book.part(i);
cout << "Part ID: " << part.partid() << endl;
cout << " Name: " << part.name() << endl;
if (part.has_vendor()) {
cout << " Vendor: " << part.vendor() << endl;
}
}
}
// Main function: Reads the entire part book from a file and prints all
// the information inside.
int main(int argc, char* argv[]) {
// Verify that the version of the library that we linked against is
// compatible with the version of the headers we compiled against.
GOOGLE_PROTOBUF_VERIFY_VERSION;
if (argc != 2) {
cerr << "Usage: " << argv[0] << " PART_BOOK_FILE" << endl;
return -1;
}
tutorial::PartDetails part_book;
{
// Read the existing data file.
fstream input(argv[1], ios::in | ios::binary);
if (!part_book.ParseFromIstream(&input)) {
cerr << "Failed to parse part book." << endl;
return -1;
}
}
ListParts(part_book);
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
#Makefile
#This creates executables for python cpp and java , to create only one language use #'make cpp' or 'make python' or 'make java'
.PHONY: all cpp java python clean
all: cpp java python
cpp: add_part_cpp list_parts_cpp
java: add_part_java list_parts_java
python: add_part_python list_parts_python
clean:
rm -f add_part_cpp list_parts_cpp add_part_java list_parts_java add_part_python list_parts_python
rm -f javac_middleman AddPart*.class ListParts*.class com/example/tutorial/*.class
rm -f protoc_middleman partbook.pb.cc partbook.pb.h partbook.pb2.py com/example/tutorial/PartBookProtos.java
rm -f *.pyc
rmdir com/example/tutorial 2>/dev/null || true
rmdir com/example 2>/dev/null || true
rmdir com 2>/dev/null || true
protoc_middleman: partbook.proto
protoc --cpp_out=. --java_out=. --python_out=. partbook.proto
@touch protoc_middleman
add_part_cpp: add_part.cc protoc_middleman
pkg-config --cflags protobuf # fails if protobuf is not installed
c++ add_part.cc partbook.pb.cc -o add_part_cpp `pkg-config --cflags --libs protobuf`
list_parts_cpp: list_parts.cc protoc_middleman
pkg-config --cflags protobuf # fails if protobuf is not installed
c++ list_parts.cc partbook.pb.cc -o list_parts_cpp `pkg-config --cflags --libs protobuf`
javac_middleman: AddPart.java ListParts.java protoc_middleman
javac AddPart.java ListParts.java com/example/tutorial/PartBookProtos.java
@touch javac_middleman
add_part_java: javac_middleman
@echo "Writing shortcut script add_part_java..."
@echo '#! /bin/sh' > add_part_java
@echo 'java -classpath .:$$CLASSPATH AddPart "$$@"' >> add_part_java
@chmod +x add_part_java
list_parts_java: javac_middleman
@echo "Writing shortcut script list_parts_java..."
@echo '#! /bin/sh' > list_parts_java
@echo 'java -classpath .:$$CLASSPATH ListParts "$$@"' >> list_parts_java
@chmod +x list_parts_java
add_part_python: add_part.py protoc_middleman
@echo "Writing shortcut script add_part_python..."
@echo '#! /bin/sh' > add_part_python
@echo './add_part.py "$$@"' >> add_part_python
@chmod +x add_part_python
list_parts_python: list_parts.py protoc_middleman
@echo "Writing shortcut script list_parts_python..."
@echo '#! /bin/sh' > list_parts_python
@echo './list_parts.py "$$@"' >> list_parts_python
@chmod +x list_parts_python
Save all these files under <proto-installation-path>/examples/ and run make
To run the example
./add_part_cpp <data-filename>
./list_parts_cpp <data-filename>
