Protocol Buffers - Data serialization

이전 블로그에서 이전 함 (원본 글 2013/04/03 작성)

Android 상에서 Native library와 Application/Service를 개발 시

상호 Layer간 data를 전달하기 위한 방법을 찾다가 발견한 data serialization 방법 중 하나.

Protobuf은은은 기존 JSON, XML에 비해 별도 parser가 필요 없고 사용하고 상대적으로 쉽다는 것과

language별 API를 제공해 준다는 것,

language별로 serialized된 data의 inteoperability가  장점으로 보임

* Data serialization 참고 (http://en.wikipedia.org/wiki/Comparison_of_data_serialization_formats)

[Protocol Buffers]

  https://developers.google.com/protocol-buffers/

What Are Protocol Buffers? Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages – Java, C++, or Python.

[Quick Example]

 . https://developers.google.com/protocol-buffers/docs/overview)

Quick Example You write a .proto file like this: message Person {   required int32 id = 1;   required string name = 2;   optional string email = 3; } Then you compile it with protoc, the protocol buffer compiler, to produce code in C++, Java, or Python. Then, if you are using C++, you use that code like this: Person person; person.set_id(123); person.set_name("Bob"); person.set_email("bob@example.com"); fstream out("person.pb", ios::out | ios::binary | ios::trunc); person.SerializeToOstream(&out); out.close(); Or like this: Person person; fstream in("person.pb", ios::in | ios::binary); if (!person.ParseFromIstream(&in)) {   cerr << "Failed to parse person.pb." << endl;   exit(1); } cout << "ID: " << person.id() << endl; cout << "name: " << person.name() << endl; if (person.has_email()) {   cout << "e-mail: " << person.email() << endl; }

사실 궁금한 것은 동일 language에서 동작하는 것 보다는 서로 다른 lanaguage에서 동작하는 지가 궁금한지라.. 

[다운로드 및 설치 방법] 

* 2.5.0 에서는 java library 빌드가 안되어 2.4.1로 빌드 및 설치

protobuf C/C++ library 빌드 및 설치 => README.txt 참고

 : (unix default) ./configure && make && make check && make install

protobuf java library 빌드 => java/README.txt 참고

 : Maven 설치 필요... sudo apt-get install maven2

 : mvn test && mvn install && mvn package

 : protobuf c/c++ library 빌디 및 설치 이후 빌드 가능

[example 빌드 및 실행]

- examples 폴더 내에서 make

 : Makefile내에서 javac_middleman 빌드 부분에서 classpath를 추가해줘야 한다.

  . cp ../java/target/*.jar .

  . Makefile에서 "javac AddPerson.java..."를 "javac -cp protobuf-java-2.4.1.jar:. AddPerson.java"로 수정

  . make

haha@cs-portable:~/source/protobuf-2.4.1/examples$ make protoc --cpp_out=. --java_out=. --python_out=. addressbook.proto pkg-config --cflags protobuf  # fails if protobuf is not installed -pthread -I/usr/local/include   c++ add_person.cc addressbook.pb.cc -o add_person_cpp `pkg-config --cflags --libs protobuf` pkg-config --cflags protobuf  # fails if protobuf is not installed -pthread -I/usr/local/include   c++ list_people.cc addressbook.pb.cc -o list_people_cpp `pkg-config --cflags --libs protobuf` javac -cp protobuf-java-2.4.1.jar:. AddPerson.java ListPeople.java com/example/tutorial/AddressBookProtos.java Writing shortcut script add_person_java... Writing shortcut script list_people_java... Writing shortcut script add_person_python... Writing shortcut script list_people_python...

Sampled proto format

protobuf-2.4.1/examples$ cat addressbook.proto // See README.txt for information and build instructions. package tutorial; option java_package = "com.example.tutorial"; option java_outer_classname = "AddressBookProtos"; message Person {   required string name = 1;   required int32 id = 2;        // Unique ID number for this person.   optional string email = 3;   enum PhoneType {     MOBILE = 0;     HOME = 1;     WORK = 2;   }   message PhoneNumber {     required string number = 1;     optional PhoneType type = 2 [default = HOME];   }   repeated PhoneNumber phone = 4; } // Our address book file is just one of these. message AddressBook {   repeated Person person = 1; }

 다른 Language에서 serialize된 stream을 parse할 수 있음.

haha@cs-portable:~/source/protobuf-2.4.1/examples$ ./add_person_cpp  Usage:  ./add_person_cpp ADDRESS_BOOK_FILE haha@cs-portable:~/source/protobuf-2.4.1/examples$ ./add_person_cpp test test: File not found.  Creating a new file. Enter person ID number: 1 Enter name: haha Enter email address (blank for none): haha@haha.net Enter a phone number (or leave blank to finish): 1111111 Is this a mobile, home, or work phone? mobile Enter a phone number (or leave blank to finish):  haha@cs-portable:~/source/protobuf-2.4.1/examples$ ./list_people_cpp test  Person ID: 1   Name: haha   E-mail address: haha@haha.net   Mobile phone #: 1111111 haha@cs-portable:~/source/protobuf-2.4.1/examples$ java -cp protobuf-java-2.4.1.jar:. ListPeople test  Person ID: 1   Name: haha   E-mail address: haha@haha.net   Mobile phone #: 1111111

일반적인 Application에서는 Native단과 Java단간의 data 교환시 정해진 data structure/object에 맞춰서 전달하겠지만 만약 JNI layer에서 공통된 format을 전달해야 한다고 한다면 protobuf를 사용해서

serialize된 data를 JNI layer를 통해 전달하고 각 layer에서 parse해서 사용한다면 간편할 듯..