Adding a HTTP Interface / REST API?

[cganas]

Hello, I want to create a crafting recipe manager web project powered by kalandralang. The challenge of working with OCAML (for us non-ML developers) could be solved with an HTTP interface to the interpreter.

For example, an incoming POST request could contain the recipe in the body of the request and the relevant command line parameters provided by query parameters. The response would be JSON with the recipe output and structured data for the costs and profits of the craft.

It looked to me that a minimal implementation would be straightforward with grinberg/opium. Any thoughts around this?

[doomeer]

An HTTP server like this is certainly doable yes. I would not host it myself though, but I assume your plan is to run it yourself just like your recipe manager itself?

Note that another possibility would be to have reverse bindings from your language to OCaml. We could in particular turn Kalandralang into a JS lib so that recipes can be executed by the browser. There are a few challenges though. For instance one needs to figure out how to manage the data files. Also I would need to learn how to make reverse bindings. Overall the HTTP server approach would be significantly simpler to implement for me.

I look forward to seeing your web project, it sounds very cool :) I'll try to implement the HTTP server.

[doomeer]

I have pushed a branch named http which provides a very rough proof of concept. You can either:

• build it with dune build and run it with _build/default/src/server/main.exe <OPTIONS>
• or build and run it in one command with dune exec kalandralang-server -- <OPTIONS>

<OPTIONS> are --port (defaults to 8080) and --data-dir. Or --help.

You can then run something like:

curl -d "$(cat examples/awakeners-orb.kld)" http://localhost:8080/run

(for some reason -d '@examples/awakeners-orb.kld' doesn't work 🤔 ) and you get a plain text response like:

--------
Marble Amulet (Rare) (Warlord / Hunter)
--------
(prefix) [T1] 15% increased Area of Effect
(prefix) [T1] Projectiles Pierce an additional Target
(prefix) [T8] Adds 3 to 7 Fire Damage to Attacks
(suffix) [T3] +41 to Dexterity
(suffix) [T5] +29% to Cold Resistance
--------
Cost:
     1 × awaken
Total: 0.99ex (175c)

Obviously a lot is missing. Can you tell me which of those features would be important for you?

• JSON output with:
  • cost (as a JSON object like {"chaos": 42, "exalt": 17.123})
  • resulting item(s) (as a string? or an array of strings for mods? or an array of something like {"kind": "prefix", "tier": 1, "modifier": "+41 to Dexterity"}?)
  • output (a string that contains what is currently served as plain text)
  • time information?
  • seed?
  • more?…
• being able to pass command-line options as query strings (which options do you need?)
• HTTPS
• being able to specify which interface (i.e. which address) to listen to
• a command-line option to specify a timeout for requests (note that this would be harder than it sounds)
• …

Of course the more features you need the more work it will be for me and I don't know when/if I'll implement them ;)

[cganas]

Hello! Thank you for taking action on this so quickly. Yes, the idea is for me to run the server. I packaged my local kalandralang copy into a ocalm/opam docker container and have it serving the HTTP interface. I was able to build a super quick PoC (kalandralang http <-> trpc + next.js) this afternoon. It's awesome seeing it work together even in this very primitive form. =)

The most important feature would be the JSON output. This would need to minimally include the cost and resulting item as structured data (something like {"kind": "prefix", "tier": 1, "modifier": "+41 to Dexterity"}). It's also important that we can incorporate a timeout for the request and handle multiple incoming HTTP requests without blocking.

A reverse binding is a really interesting option with the added benefit of bringing the interpreter client side. However, as you pointed out, this would require a lot of upfront work. I think we can take a deeper look into that later if there is excessive challenge with the HTTP approach.

I will continue to work on refining the idea and building out the PoC so I can get an accurate picture of what features would be necessary to bring things to life. My current motivation is to setup an archive of crafting recipes for items and assign them to a character. The crafting list across all remaining-to-be-crafted recipes for that character could then be used to determine the total expected costs and raw materials to finish the items. It would bring a whole new level of accuracy to a low budget or high budget build! If anyone has contributing thoughts or experience, please let me know.

[doomeer]

Very nice! Thanks for the screenshot :) Looks cool! And indeed it sounds quite useful to be able to evaluate the budget of a build at any time with real market values.

I have pushed a new version of the branch. It now serves JSON instead of plain text. I think you'll find all you need in the data but don't hesitate to tell me if you miss anything. I didn't document the format but I'm sure you'll have no trouble reading the output without documentation, it's pretty self-explanatory.

I tested and it appears that one can already perform multiple queries in parallel. The library I use (cohttp) probably uses threads I guess. However, I didn't implement a timeout yet. I think it should actually be easier than I thought at first though.

[doomeer]

@cganas are you interested in me rebasing the http branch on top of master to get the latest changes (mostly related to the exalt/divine swap) or have you stopped working on this project? (I wouldn't blame you, passion projects come and go for everyone :D )

[AR-234]

I have a simular request, want to build some tooling around my recipies: could it be possible to add the json output as a flag to main.

people then could use what ever language or tools they want to integrate it into there stuff.
they only need to build a wrapper around the application (or maybe even with the http server as an additional flag)

[doomeer]

I have rebased the http branch. Some fields changed name because of the ex <-> div swap.

I have added a commit on top of it to add a --json command-line option. This causes Kalandralang to output the same results as the HTTP server would, except that the logs field is empty. Instead, logs are still available, but on stderr instead of stdout. Example:

$ kalandralang run examples/unveil.kld --json 2> logs
{
  "count": 1,
  "items": [],
  "average_paid": {
    "suffixes_cannot_be_changed": 1,
    "aisling": 1
  },
  "average_gained": {},
  "average_paid_div": 2.484819101872614,
  "average_gained_div": 0,
  "logs": null
}
$ cat logs
Loading kalandralang.cache...
Loading costs.json...
Ready.
12.397% (prefix) 17-20% increased Projectile Damage
                 23-25% increased Projectile Speed
12.397% (prefix) 17-20% increased Melee Damage
                 +2 to Melee Strike Range
9.917% (prefix) +51-55 to maximum Mana
                6-7% reduced Mana Cost of Skills
9.917% (prefix) +51-55 to maximum Mana
                7-8% of Damage taken Recouped as Mana
9.917% (prefix) +51-55 to maximum Mana
                Regenerate 5.3 Mana per second
16.529% (prefix) +55-60 to maximum Mana
                 Regenerate 33.3 Life per second
16.529% (prefix) +55-60 to maximum Life
                 Regenerate 5.3 Mana per second
12.397% (prefix) 17-20% increased Area Damage
                 14-16% increased Area of Effect
--------
Agate Amulet (Rare)
--------
(prefix) {crafted} Suffixes Cannot Be Changed
(prefix) [T1] 20% increased Area Damage
              15% increased Area of Effect
(suffix) [T1] +53 to Dexterity
(suffix) [T1] +51 to Intelligence
(suffix) [T1] +53 to Strength
--------
Cost:
     1 × aisling
     1 × suffixes_cannot_be_changed
Total: 2.48div (410c)

@AR-234 I think this should suit your needs. It is possible that there are still some bugs where the program would print stuff on stdout, breaking the JSON output. If you get a JSON parse error, it may be because of such a bug.

This is still very experimental and barely tested, so it'll stay in the http branch for now =)