myrpc documentation · 2019. 4. 2. · myrpc documentation, release 0.0.4 •return value is...

MyRPC DocumentationRelease 0.0.4

Szalai András

May 12, 2014

Contents

i

MyRPC Documentation, Release 0.0.4

Contents:

Contents 1


2 Contents

CHAPTER 1

Features

Short summary of MyRPC features:

• No external dependencies.

• Cross-platform capability.

• IDL-based client and server stub generation.

• Binary capable (no need for escaping of binary data).

• Single roundtrip protocol, ideal for HTTP (but no limited to).

• Support various data types: string, binary, signed and unsigned integers, floating point, list, structure and enu-meration.

• All data types are supported on all platforms.

• Support exceptions.

• Correct input validation of the received messages.

• Legacy free code (since we are new :).

1.1 Target languages

MyRPC supports the following target languages:

Lan-guage

Generatorname

Client stubgeneration

ClientAPI

Processor stubgeneration 4

ProcessorAPI 5

Python py Yes Sync Yes Sync andAsync

JavaScript6

js Yes Async Yes Sync andAsync

1Also known as server stub. In MyRPC terminology it is called processor stub.2In case of HTTP, asynchronous processor API can be used to implement long polling.3Node.js and browsers are both supported.4Also known as server stub. In MyRPC terminology it is called processor stub.5In case of HTTP, asynchronous processor API can be used to implement long polling.6Node.js and browsers are both supported.

3

http://en.wikipedia.org/wiki/Push_technology#Long_pollinghttp://en.wikipedia.org/wiki/Push_technology#Long_polling


4 Chapter 1. Features

CHAPTER 2

Installation

MyRPC source distribution consists of several components. They can be classified into two main categories:

• myrpcgen: MyRPC code generator tool, used to compile MyRPC IDL files to target language dependent sourcecode.

• Runtime libraries: they are required to compile and execute code which is generated by myrpcgen.

MyRPC is highly modular, so myrpcgen and target language dependent runtime libraries can be installed and usedseparately. The only constraint we have is: myrpcgen version and runtime library versions must match. For example,if we have some generated code which are generated with myrpcgen version 1.2.3, then a runtime library which hasthe same version (1.2.3) must be used.

The latest stable release of MyRPC is 0.0.4.

2.1 MyRPC code generator tool

Requirement:

• Python >= 3.3.

Source distribution can be downloaded from https://pypi.python.org/pypi/myrpcgen.

After unpacking the source distribution, change directory to myrpcgen-x.x.x. Here you can do system-wideinstallation (as root):

python setup.py install

or, as an alternative solution, myrpcgen can be installed inside the user’s home directory:

python setup.py install --user

For more information about Distutils based installation, see http://docs.python.org/3.3/install/index.html.

MyRPC code generator tool will be installed into prefix/bin, be sure to put this directory in your PATH.

2.2 Python runtime

Requirement:

• Python >= 3.3.

5

https://pypi.python.org/pypi/myrpcgenhttp://docs.python.org/3.3/install/index.html


Source distribution can be downloaded from https://pypi.python.org/pypi/myrpc-runtime.

The installation steps are similar to what is described in MyRPC code generator tool.

2.3 JavaScript runtime

2.3.1 Browser

Browser-side runtime source code can be downloaded directly from our repo at https://github.com/bandipapa/MyRPC.Go to releases, and select a release (rel-x.x.x tags) which matches your myrpcgen version. Source code is availableunder runtime/js/src. (In the future, we will prepackage browser-side runtime sources for each release.)

2.3.2 Node.js

Requirements:

• Node.js >= 0.10.26.

• npm >= 1.3.11.

The installation is very simple, just enter the following command at your project root directory:

npm install myrpc-runtime

or, if you want system-wide installation, then pass -g to npm install. It is also possible to install a given version, usingthe @ syntax:

npm install [email protected]

2.4 Examples

Requirements:

• In general, GNU Make >= 3.82 is required for building the examples.

• There can be additional, example specific dependencies, see Examples.

Source code can be accessed at https://github.com/bandipapa/MyRPC/tree/master/examples.

6 Chapter 2. Installation

https://pypi.python.org/pypi/myrpc-runtimehttps://github.com/bandipapa/MyRPChttps://github.com/bandipapa/MyRPC/tree/master/examples

CHAPTER 3

IDL introduction

In this section, we will cover MyRPC IDL syntax and basic myrpcgen (MyRPC code generator tool) usage withincluded examples.

3.1 Namespace

Let’s start with a simple calculator example, create an empty textfile called calculator.idl. Because MyRPCgenerates code, it needs some guidance which namespace the generated stuff will go into:

namespace py CalculatorService

Explanation:

• In most cases, one namespace declaration is required per target language we would like to generate code for.

• py is the name of target language (in this case it is Python). In MyRPC terminology it is called generator. Foravailable generators, see Target languages.

• CalculatorService is the name of Python package where the generated files will go into. Actually, the interpre-tation of namespace is generator dependent, see Generator specific information.

3.2 Method

Our first method will calculate the sum of two integers:

beginmethod sumin 0 required i32 ain 1 required i32 bout required i32

endmethod

Explanation:

• beginmethod declares a method called sum.

• Methods can have zero or more input arguments, which are declared with in keyword. There are two inputarguments: a and b. Both are required, meaning that they can’t be null. If we allow null values, then optionalkeyword can be used instead. i32 is the data type of arguments (32 bit signed integer). Integer numbers rightafter in are called field identifiers, or fids. During (de)serialization, MyRPC uses fids (0 and 1) instead of names(a and b) to identify input arguments. Fids must be unique and be between 0 ... 216 - 2.

7


• Return value is specified by out. In method declaration, exactly zero or one out keyword is allowed. If we don’thave out, the method doesn’t return value.

In place of i32, we can use any other primitive or user-defined type (excluding exceptions), provided they are declaredbefore. This is also true for all declarations and keywords where data type expected. For more information on availabletypes, see Type mapping.

3.3 List

To allow variable number of input arguments, we can write generalized version of the sum method by using list insteadof separate a and b arguments:

list IntegerList i32

beginmethod sum_listin 0 required IntegerList in_listout required i32

endmethod

Explanation:

• The list keyword is used to define new list type, called IntegerList.

• List elements have i32 data type. Elements in list can’t be null (MyRPC doesn’t support it).

• The newly created type (IntegerList) can be referenced later, where data type expected.

3.4 Enumeration

We can extend our IDL with a method, which classifies input argument:

beginenum IntegerTypeentry IS_ZEROentry IS_NEGATIVE -1entry IS_POSITIVE 1

endenum

beginmethod classifyin 0 required i32 aout required IntegerType

endmethod

Explanation:

• Enumerations are started using beginenum declaration.

• One entry declaration is required per enumeration entry (or member).

• Enumeration entries can have an optional integer value (-1 and 1). If the value is not specified, then it will havethe value of previous entry value + 1. The first entry of the enumeration has the default value of 0.

3.5 Structure

The next one is a “batch calculator” example:

8 Chapter 3. IDL introduction


beginenum OperationTypeentry ADDentry SUBSTRACTentry MULTIPLYentry DIVIDE

endenum

beginstruct Operationfield 0 required OperationType op_typefield 1 required float afield 2 required float b

endstruct

list OperationList Operationlist ResultList float

beginmethod batch_calculatein 0 required OperationList op_listout required ResultList

endmethod

Explanation:

• beginstruct declares a structure called Operation.

• Field declarations inside structures are similar to argument declaration of methods, but instead of in and out, wehave to use field keyword here.

• On instantiation, all structure fields will be set to null by default.

3.6 Exception

Methods can throw exceptions. Exceptions typically represent some error condition. Here is how you can declarethem:

beginexception DivideByZeroExceptionendexception

beginexception OtherExceptionfield 0 optional string message

endexception

beginmethod dividein 0 required float ain 1 required float bout required floatthrow DivideByZeroExceptionthrow OtherException

endmethod

Explanation:

• beginexception declares exceptions called DivideByZeroException and OtherException.

• Exception declarations have the same syntax as structure declarations (see above).

• Zero or more throw keywords inside beginmethod, each declaring what to throw.

• On instantiation, all exception fields will be set to null by default.

3.6. Exception 9


3.7 Comment

The hash character (#) is used to start a comment in MyRPC IDL.

3.8 Invoke myrpcgen

Let’s say, we have finished IDL and we would like to generate Python processor stub. To do it, execute:

myrpcgen -g py -d gen-py -P calculator.idl

Explanation:

• -g: specifies which generator to use. The namespace declaration is consulted to determine target languagenamespace, see Namespace.

• -d: output directory.

• -P: says that myrpcgen will generate processor stub. The opposite option would be -C, to generate client stub.

10 Chapter 3. IDL introduction

CHAPTER 4

Type mapping

The MyRPC IDL supports the following types, together with their language mappings:

Description MyRPC Python JavaScriptNull value None nullBinary buffer binary bytes Uint8ArrayString string str StringBoolean bool True, False true, falseUnsigned integer, 8 bit ui8 int NumberUnsigned integer, 16 bit ui16 int NumberUnsigned integer, 32 bit ui32 int NumberUnsigned integer, 64 bit ui64 int Number 3

Signed integer, 8 bit i8 int NumberSigned integer, 16 bit i16 int NumberSigned integer, 32 bit i32 int NumberSigned integer, 64 bit i64 int Number 2

Single precision floating point float float NumberDouble precision floating point double float NumberEnumeration enum class 4 Object 2

List list list ArrayStructure struct class 1 Object 2

Exception exception class 1 Object 2

Binary buffer, string, boolean, integer and floating point types are primitive types. The remaining ones are user-definedtypes (except the null value).

1See JavaScript for more details.2See Python for more details.3See JavaScript for more details.4See Python for more details.

11


12 Chapter 4. Type mapping

CHAPTER 5

Generator specific information

5.1 Python

5.1.1 Namespace

Python generator interprets namespace IDL declaration as a package name. If we have the following in IDL:

namespace py TopPackage.SubPackage

then the following files will be created in the directory specified with the -d outdir option of myrpcgen:

File Descriptionoutdir/TopPackage/__init__.pyEmpty file.outdir/TopPackage/SubPackage/__init__.pyEmpty file.outdir/TopPackage/SubPackage/Types.py(De)serializers for: method arguments, return values and for user-defined types.

All the user-defined types will be in the TopPackage.SubPackage.Types module.outdir/TopPackage/SubPackage/Client.pyClient stub implementation (Client class) in the TopPackage.SubPackage.Client

module.outdir/TopPackage/SubPackage/Processor.pyProcessor stub implementation (Processor class) and abstract interface

(Interface class) in the TopPackage.SubPackage.Processor module.

5.1.2 Enumeration

If we have the following in IDL:

beginenum Statusentry OKentry FIRST_ERRORentry SECOND_ERROR 42

endenum

The generated code will look like (Types.py):

class Status:OK = 0FIRST_ERROR = 1SECOND_ERROR = 42

13


5.1.3 Structures and exceptions

MyRPC generated structure and exception classes behave exactly the same, with one exception: exception classeshave Exception as their parent class, but structures don’t have parent class.


beginstruct UserInfofield 0 required string username...

endstruct

beginexception SizeTooLargefield 0 required ui16 maxsize...

endexception

UserInfo and SizeTooLarge can be instantiated as the following:

from TopPackage.SubPackage.Types import UserInfo, SizeTooLarge

obj = UserInfo()exc = SizeTooLarge()

Depending on -f option of myrpcgen, fields can be accessed as shown here:

Optionvalue

Getter invocation Setter invocation

direct value = obj.username, value = exc.maxsize obj.username = value, exc.maxsize = valuecapital value = obj.getUsername(), value =

exc.getMaxsize()obj.setUsername(value),exc.setMaxsize(value)

underscore value = obj.get_username(), value =exc.get_maxsize()

obj.set_username(value),exc.set_maxsize(value)

5.2 JavaScript

The following parts describe browser-side behaviour. For Node.js specific information, see Node.js.

5.2.1 Namespace


namespace js TopNS.SubNS

then the following files will be created in the directory specified with the -d outdir option of myrpcgen:

File Descriptionoutdir/Types.js(De)serializers for: method arguments, return values and for user-defined types. All the

user-defined types will be in the TopNS.SubNS.Types namespace.outdir/Client.jsClient stub implementation in TopNS.SubNS.Client class.outdir/Processor.jsProcessor stub implementation in TopNS.SubNS.Processor class.

14 Chapter 5. Generator specific information


5.2.2 64 bit unsigned and signed integers

Integers in JavaScript are limited to -253 ... 253, however MyRPC runtime doesn’t check range limitation. For moreinformation, see http://ecma262-5.com/ELS5_HTML.htm#Section_8.5.

5.2.3 Enumeration


beginenum Statusentry OKentry FIRST_ERRORentry SECOND_ERROR 42

endenum

The generated code will look like (Types.js):

TopNS.SubNS.Types.Status = {OK: 0,FIRST_ERROR: 1,SECOND_ERROR: 42

};

5.2.4 Structures and exceptions

MyRPC generated structure and exception objects behave exactly the same.


beginstruct UserInfofield 0 required string username...

endstruct

beginexception SizeTooLargefield 0 required ui16 maxsize...

endexception

UserInfo and SizeTooLarge can be instantiated as the following:

var obj = new TopNS.SubNS.Types.UserInfo();var exc = new TopNS.SubNS.Types.SizeTooLarge();

Depending on -f option of myrpcgen, fields can be accessed as shown here:

Optionvalue

Getter invocation Setter invocation

direct value = obj.username, value = exc.maxsize obj.username = value, exc.maxsize = valuecapital value = obj.getUsername(), value =

exc.getMaxsize()obj.setUsername(value),exc.setMaxsize(value)

underscore value = obj.get_username(), value =exc.get_maxsize()

obj.set_username(value),exc.set_maxsize(value)

5.2. JavaScript 15

http://ecma262-5.com/ELS5_HTML.htm#Section_8.5


5.2.5 Node.js

Using the --js_target node option of myrpcgen, you can request code generation for Node.js. The generatedcode will have the following differences compared to browser-side code:

• Instead of global symbols, we are using the module concept of Node.js, therefore namespace declaration isignored.

• Types module can be accessed from Node.js:

var Service = require("outdir/Types");var Types = Service.Types;

var obj = new Types.UserInfo();

16 Chapter 5. Generator specific information

CHAPTER 6

Runtime API

6.1 Python

FIXME: To be written.

6.2 JavaScript


17


18 Chapter 6. Runtime API

CHAPTER 7

Input validation


19


20 Chapter 7. Input validation

CHAPTER 8

BinaryCodec implementation notes

BinaryCodec has the following implementation details and limits:

• Message type is 8 bits.

• Field identifiers are 16 bits.

• Data type is 8 bits.

• Maximal size of binary and string (in encoded format) is 232 - 1 bytes.

• Maximal number of list elements is 232 - 1.

• Enumerations are represented by 32 bit signed integers.

• Numbers are transmitted in network byte order.

21


22 Chapter 8. BinaryCodec implementation notes

CHAPTER 9

Examples

Here you can find some information about demo code included in MyRPC source distribution.

9.1 Gallery

Source code: https://github.com/bandipapa/MyRPC/tree/master/examples/gallery.

This is an example demonstrating how you can use MyRPC to create a simple web/ajax-based picture gallery applica-tion, with Python server and JavaScript client code. Try to upload an image and see what happens... :)

Requirements:

• Pillow >= 2.3.0 (http://python-imaging.github.io)

To compile MyRPC IDL files:

cd examples/gallerymake

To start the demo:

python gallery 8080

where 8080 is a portnumber, feel free to use any other.

Then point your browser at http://yourip:8080, and upload some images. The uploaded images are simply stored inmemory (not saved to disk).

Source code files:

File Descriptiongallery Contains a standalone webserver and the implementation of GalleryService. Most of the

code deals with the HTTP infrastructure & static file serving, the MyRPC relevant partsare: GalleryServiceImpl class and handle_rpc method.

gallery.idl Interface description of GalleryService.GalleryService/*.pyGenerated code (stubs) for Python server.www/index.html,www/gallery.js

HTML/JavaScript webcontent, served by the built-in webserver.

www/gen/*.js Generated code (stubs) for JavaScript client.www/myrpc/*.js MyRPC JavaScript runtime libraries (these are just symlinks).

23

https://github.com/bandipapa/MyRPC/tree/master/examples/galleryhttp://python-imaging.github.iohttp://yourip:8080


9.2 Chat

Source code: https://github.com/bandipapa/MyRPC/tree/master/examples/chat.

This is an example demonstrating how you can use MyRPC to create a simple real-time web application in pureJavaScript (browser-side and Node.js server) based on asynchronous MyRPC processor API.

To compile MyRPC IDL files:

cd examples/chatmake

To start the demo:

node chatserver.js 8080

where 8080 is a portnumber, feel free to use any other.

Then point your browser at http://yourip:8080, and start chatting.

24 Chapter 9. Examples

https://github.com/bandipapa/MyRPC/tree/master/examples/chathttp://yourip:8080

CHAPTER 10

Bug reporting

Bug tracking system can be found at: https://github.com/bandipapa/MyRPC/issues.

25

https://github.com/bandipapa/MyRPC/issues


26 Chapter 10. Bug reporting

CHAPTER 11

License

Copyright (c) 2014, Szalai András, All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that thefollowing conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the followingdisclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the follow-ing disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANYEXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIESOF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENTSHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, IN-CIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITEDTO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSI-NESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CON-TRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANYWAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAM-AGE.

27

myrpc documentation · 2019. 4. 2. · myrpc documentation, release 0.0.4 •return value is...

Documents