Re: ocamldoc generation and packed files

[Thomas Gazagnaire <thomas.gazagnaire@xxxxxxxxx>]

I don't have concrete available solution for the .annot files. The two I can 
see are :

* releasing the .ml -> .annot part of the compiler as a stand-alone tool and 
use it to generate individual .annot files (I guess that's not very different 
from running ocamlc -annot on every file)
* having a tool to split out a packed .annot file into individual .annot files. 
I guess it's not so difficult to do (just need to parse the directive lines in 
the packed source file + its annot file and do some comparison), but we don't 
have it yet.

Thomas

On Aug 19, 2011, at 11:58 AM, Anil Madhavapeddy wrote:

> As a followup to this, I've integrated ocaml-ocp into my tree and it's 
> working really well. Thanks for releasing this Thomas!  There was an 
> off-by-one in the cpp directives for line numbering (fixed in our tree).
> 
> It would be very useful to have some way for .annot files to work without 
> compiling the individual sub-files; any thoughts on how this might work?
> 
> Anil
> 
> On 13 Aug 2011, at 20:39, Anil Madhavapeddy wrote:
> 
>> Sure, go for it if you get a chance.
>> 
>> The current packing "works", but the doc target cannot be built at the same 
>> time as the normal code (because it causes the source pack to leave .ml 
>> files in lib/std/foo.ml, which then causes the lib/std/foo.cm{xo} to be 
>> rebuilt instead of the -packed copied .cmx to be used).
>> 
>> Basically, its a bit of a mess :-)
>> 
>> As a sneak preview, check out http://github.com/avsm/mirage-browser.git (on 
>> index.html). This is a *very* rough jQuery-based module browser of the 
>> Mirage OCamldoc that I'd like to have live on the website before the CUFP 
>> tutorial.  The live search is pretty nice, and I'll flesh out the rendering 
>> over the next few days.
>> 
>> This tool is also quite standalone as it uses the output of 
>> odoc_json/ocamldoc, and so could be applied to the standard distribution and 
>> also to the Citrix xapi-libs (if you're interested, David). 
>> 
>> Killer feature: the js_of_ocaml interactive top-level should work in here... 
>> :-)
>> 
>> Anil
>> 
>> On 13 Aug 2011, at 17:50, Thomas Gazagnaire wrote:
>> 
>>> No ocamlbuild integration yet but would be definitely useful to add. I
>>> can have a look at integrating it to mirage build next week if it's
>>> useful.
>>> 
>>> --
>>> Thomas
>>> 
>>> 2011/8/13 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>> Thanks Thomas, that looks very useful and definitely better than the 
>>>> script in tree at the moment.
>>>> 
>>>> One question: ocp-pack also requires that the list of files be passed in 
>>>> dependency order, or else the resulting pack file will not compile.  Have 
>>>> you tried to integrate it as an ocamlbuild rule, so that it can 
>>>> automatically sort the input modules before generates the packed file / 
>>>> functor?
>>>> 
>>>> Anil
>>>> 
>>>> On 12 Aug 2011, at 20:01, Thomas Gazagnaire wrote:
>>>> 
>>>>> It will be anounced quite shortly on the ocaml mailing list, so I can
>>>>> give you the link to ocp-pack :
>>>>> 
>>>>> http://www.ocamlpro.com/code/2011-08-10-ocaml-pack-functors.html
>>>>> http://www.ocamlpro.com/files/ocp-pack-1.0.0.tar.gz
>>>>> 
>>>>> It should help you to pack modules easily and have a working doc.
>>>>> 
>>>>> --
>>>>> Thomas
>>>>> 
>>>>> 2011/7/28 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>>>> That tool will be very useful! I've committed the 'make doc' target. The 
>>>>>> output is still pretty dirty as we don't use the documentation tags 
>>>>>> properly, but I'll go through adding .mli files and adding proper 
>>>>>> documentation on the more stable interfaces.
>>>>>> 
>>>>>> It would be quite nice to eventually have a single documentation output 
>>>>>> for all the backends, with an addition section saying 'only present in 
>>>>>> Xen' or 'only present in Node'. I think that should be possible by 
>>>>>> parsing the ocamldoc dump outputs, but something for the future!
>>>>>> 
>>>>>> Anil
>>>>>> 
>>>>>> On 27 Jul 2011, at 16:56, Thomas Gazagnaire wrote:
>>>>>> 
>>>>>>> We have as well a program which pack together ML files (and is able to 
>>>>>>> functorize packs as well...) it is not released yet, but I guess we can 
>>>>>>> open-source it shortly.
>>>>>>> 
>>>>>>> But I think overriding the default rule for ocamldoc+pack in ocamlbuild 
>>>>>>> is sufficient for now on so you should push your patch :-)
>>>>>>> 
>>>>>>> Thomas
>>>>>>> 
>>>>>>> On Jul 27, 2011, at 5:36 PM, Anil Madhavapeddy wrote:
>>>>>>> 
>>>>>>>> I'd really like to generate ocamldoc (HTML/PDF) of all the various 
>>>>>>>> libraries so that it's easier to learn Mirage (and support editor 
>>>>>>>> auto-completion, etc).
>>>>>>>> 
>>>>>>>> The big problem is that ocamldoc doesn't support packed modules, and 
>>>>>>>> we use packing quite extensively (in Net, Http, Block, etc).
>>>>>>>> 
>>>>>>>> So I've hacked up an ocamlbuild target that concats together the 
>>>>>>>> *source* ML files from an .mlpack and uses that to generate the 
>>>>>>>> ocamldoc for the standard library, with one set of HTML files 
>>>>>>>> generated per backend (Xen, Net-Direct, Net-Socket, and so on).
>>>>>>>> 
>>>>>>>> The issue is that these rules are a little grim: everywhere where we 
>>>>>>>> have a .mlpack file at the moment, we need to override that rule to 
>>>>>>>> generate a concatenated ML file that is used for ocamldoc (but not for 
>>>>>>>> actual compilation, since line numbers get lost since those aren't 
>>>>>>>> preserved when converting from ML files into a single big one).
>>>>>>>> 
>>>>>>>> Does anyone have a better solution for ocamldoc and packed files? All 
>>>>>>>> of the grimness here is hidden away in the ocamlbuild rules, so I'm 
>>>>>>>> inclined to just commit this patch, and perhaps see about adding -pack 
>>>>>>>> support into ocamldoc at a later stage (there's an open bug in Mantis 
>>>>>>>> somewhere).
>>>>>>>> 
>>>>>>>> Dave, do you use pack in XAPI, and/or ocamldoc?  I wonder if everyone 
>>>>>>>> else (like Core) also have their own swanky 'cat ML files into one' 
>>>>>>>> script too...
>>>>>>>> 
>>>>>>>> -anil
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>