Over the last few posts (here and here) I’ve given a few hints as to compiling for other platforms. Basically, this all relates to how well protobuf-net works on platforms like iOS, WinRT/Metro, Silverlight, Phone 7, etc. These heavily restricted runtimes don’t allow much meta-programming, and they might be running on low-power CPUs, so reflection (even if possible) is not ideal.
I’ve played with assembly generation before, but with mixed results. For example, here for Phone 7. This could just about work for some frameworks, but was patchy on some, and won’t work at all for others.
Well, all that IKVM shininess has opened up a whole new set of tools. The small beauty that I’m ridiculously pleased with is a new utility exe in the SVN trunk (will be part of a proper release soon): precompile.
This unassuming little tool works a bit like “sgen”, but with the ability to target multiple frameworks. What it does is:
- inspect the input assembly (or assemblies) to resolve (if it can) the target framework
- initialize an IKVM universe targeting that framework
- load the core framework libraries, protobuf-net, and the input assemblies into the IKVM universe
- scan the input assemblies for types marked [ProtoContract]
- add those to a protobuf-net model
- compile the model to a type / assembly of your choosing
To use that in a project you might:
- create a new DTO project in your chosen framework, and compile it
- execute precompile to generate to a serializer assembly
- from your application project, reference the DTO and serializer assemblies
- use the type you created
For example, say I create a new Phone 7 DTO assembly, called PhoneDto (because it is late and I lack imagination). I can then create a serialization assembly via:
precompile {some path}\PhoneDto.dll –o:PhoneSerializer.dll –t:MySerializerThis will generate a library called PhoneSerializer.dll, which you can reference from your main project (in addition to the DTO and the protobuf-net core).
Then, just use MySerializer:
var ser = new MySerializer();
ser.Serialize(output, obj);
I hope this finally solves a number of tooling issues. I’m pretty pleased with it. I’ve tested it against a range of different frameworks, and it has worked well – but if you get problems, just let me know (comment here, or email me, or log an issue on the protobuf-net project site).
22 comments:
Could this tool help me precompile a RuntimeTypeModel that is created via code (rather than attributes)?
The RuntimeTypeModel.Compile(string, string) method provides a way to compile a RuntimeTypeModel into a file, but I don't see a straightforward way to do it in the build process (e.g. Post-build event).
If indeed the precompiler only support attribute-based discovery, perhaps you could add another attribute [PrecompileRuntimeTypeModel] that can be placed on a public static method with a return type of RuntimeTypeModel, and the precompiler would invoke that method, compile the model that it returns and include it in the output assembly.
@allon targetting which framework? If regular .NET, I'd write a console exe that sets up the RuntimeTypeModel and calls Compile(string,string), and call that exe in the build process. If you are targetting another platform, it gets *much* more involved.
Yes, vanilla .NET. I just thought the protobuf-net precompiler could be that tool, rather than me (and probably others) having to write one.
At the moment, the "custom model" approach isn't supported by the precompiler - there's technical reasons behind that.
But by comparison: sgen doesn't let you define XmlAttributeOverrides at runtime *at all*. So having the *ability* to write a console exe *of whatever complexity you like* that builds a model then calls ".Compile(...)" is still pretty cool, IMO.
Yep, it sure is!
excuse me, I want a copy of this tool for the development of windows phone 7. could you send to fanzhaoxing@gmail.com ?
excuse me, could you send a copy of the tool to me(fanzhaoxing@gmail.com)? I am busy developing windows phone 7 applications. thanks
@Felven This is fully available in r565: http://code.google.com/p/protobuf-net/downloads/list
(note: that is a few commits behind, which relate to the portable class library framework; if you're using portable class library, let me know and I'll update the download)
This is a precious little gem hidden far too deep in the interwebs, in my opinion.
I am developing an OSM vector map viewer for WP7, and thus need to load the map at runtime on the phone.
Long story short, a map load time of ~35 seconds (DataContractSerializer) has been reduced to approx 2 seconds. Of course the 10x space reduction is nice too.
Many thanks for this! Now I do not have to spend a week on developing my own binary format.
Thanks for all your great work, Marc. One question, does the precompiler look for [DataContract] and [DataMember(Order=n)] as well as [ProtoContract]?
hi Marc,
how do you thing about building a portable version of protobuf-net assembly? which can use both in projects using windows phone, silverlight and server, mean we can now write only 1 assembly for message contracts, ser/deserialize helpers and re-used it in all projects of solution.
@tào lao the r580 download on google-code already includes portable library builds for both Full are CoreOnly. HOWEVER! I will advise that this has many compromises (due to needing to satisfy all frameworks), and the framework-specific builds are preferred.
I have some problem in baseclass and subclass ,i want my dll can use in ios,but when i use attributes(ProtoInclude) there is an error "Cyclic inheritance is not allowed" ,so i need find another way to precompile my dll, i set ProtoMember's "DynamicType= true" and drop ProtoInclude. The dll precompile success. but when i use my dll to Serializer my class there is an runtime error "Unexpected sub-type".Could you give me some help?Thank you,and my english is poor...
Hi,
could you give some example how to use precompile tool on windows phone 7.1 using as input .proto files ?
I need to use .proto, because it allows for same message definition for multiple languages.
Maybe it is obvious, but I am new to .NET and the documentation is 'a bit' lacking in information.
Starting from .proto, it will be a couple of steps:
0: create a WP7 class libray, called MyDtoLibrary (or whatever you want; the point is: this library is for your DTOs) - and reference protobuf-net from this library
1: use "protogen" to generate C# from the .proto
2: include the generated C# into your library project, MyDtoLibrary
3: build the MyDtoLibrary project
4: point "precompiler" at the newly compiled MyDtoLibrary.dll, outputting something like "MySerializer.dll" (again, do what you like with the names)
5: in your app, reference MyDtoLibrary, protobuf-net, and MySerializer.dll - use the generataed serializer from there (do not use Serializer.Serialize, etc)
Excuse me. I found that it cannot precompile an generic object like
[ProtoContract]
public class Result
{
}
in CF 3.5. It catch exception abt
"object reference not set to an instance of an object"
should be
public class Result<T>
{
}
Hi Marc,
Thanks for posting such an amazing blog.
We are facing huge perfomance issues in my firm due to serializaiton. The problem is serializing between dataset to xml to strongly typed c# object. The solution seems to be an xml serializer which is capable of providing interop between xml and wcf serialization.
> Is protobuf-net capable of doing that?
> Do you know any open source xml serializer?
Please advice.
Thanks
@anon protobuf uses a defined binary format; it is not compatible with XML. You could try ServiceStack for that, but I can't be sure.
How would you go about precompiling a RuntimeTypeModel DLL for (say) WP7. Is it possible (I guess by possible I really mean reasonably feasible)
@David what do you have at the moment? If you have a self-describing model, the standalone tool will do the job. If your intent is to configure it manually: that is trickier. I could probably prepare an example if you have a concrete example.
Hi,
The objects I want to serialize are in code I can't (easily) apply attributes to so I've been using the RunTimeTypeModel. The (not complete) code I have is in this thread OSMSharp.
It's hopefully not essential I precompile it but any performance gains is a good thing
Post a Comment