Version 1.94 of SPJS released.

Version 1.94 of SPJS released.

Grbl buffer flow adding the following strings jumping the buffer to front of line. “[!~?]|(\u0018)|[\u0080-\u00FF]”
TinyG G2 buffer max changed to 250 from 100.
Added TCP server for Cayenn.
Added mutex lock in serial port opening to alleviate some deadlocking on opening of ports.

@Justin_Adie and @Luca_Nervetti deserve credit for Grbl updates.
https://github.com/chilipeppr/serial-port-json-server/releases/tag/v1.94

@jlauer
Hi @Luca_Nervetti and I have been discussing this code and the buffer usage for grbl 1. one thing that I am struggling to understand is why the buffersize is compared against the entire size of the command queue.

In grbl v1, the number of bytes left in the RX Buffer is transmitted in a near realtime basis. So it feels like a more streamlined solution to decide whether to send the next item in the queue based on whether there is available space in the buffer.

I cannot see any method in the queue class to get the length of a particular item. So would you recommend that the methodology would be

var msg, _ = b.q.peek();
if (len(msg) > b.availableBuffer){
//halt the queue
} else {
//send message
_, _ = b.q.poll();
}
(also adding some math to adjust availableBuffer until the next definitive update from grbl is received).

or is there a more elegant solution? or is there a different reason why the length of all the queue is compared against the maximum buffer size?

doing the latter is definitely causing some issues. For example, just refreshing the config causes potentially significant corruption as SJPS does not stop sending into the buffer, but the eeprom write command takes a while for each line. This is bad enough so that we’ve seen corruption even with two config changes being sent sequentially.

jic you need here is a binary for the 1.94 source on OSX: https://www.dropbox.com/s/tlqvnif5ep29ubh/serial-port-json-server?dl=0

one thing to note - to kill the server you have to CTRL+C. you can stop the process with CTRL+Z but to detach the port you will need to kill the process stub with kill -s 9

Feel free to rewrite anything you want in that buffer, but I do think that the way it works today is really good other than the possible deadlocking due to the threading issue. It is way safer to have the SPJS buffer calculate it’s own character count rather than wait for a roundtrip number back from Grbl. The reason is that the roundtrip delay on getting back a new count, could actually mean that you’ve sent a 2nd or 3rd or even 4th line into Grbl during that time and way overflowed the buffer. You can change to a single threaded model and send a line, then wait for the return count, and then send the next line, but what you’ll get is stuttering on small Gcode moves like on corners because the cnc doesn’t have enough lines in the planner.

You had stated "decide whether to send the next item in the queue based on whether there is available space in the buffer. " and that is in fact 100% how the SPJS buffer flows work.

On your question of “reason why the length of all the queue is compared against the maximum buffer size” the b.q. is the list of commands that have been sent to Grbl (i’m pretty sure anyway, but not 100%) that we’re waiting on an OK response for. So you need to compare that to the max size of 128.

As for your EEPROM problem, you can add a pause on send commands in ChiliPeppr, which is how TinyG does it from it’s configuration widget because the same issue exists there. This was something solved at a higher layer in the food chain in the TinyG workspace. I wouldn’t blame the buffer algorithm on this one.

the pause is added in the workspace Luca and I are working on. so we have solved it at the UI level, but that’s the wrong place architecturally: the communication layer should take responsibility for ensuring that the buffer does not get overflowed.

So my working assumption is that this is not currently happening correctly in the grbl part of SPJS. I had assumed that the queue received all incoming messages due to be sent to the 'duino. If actually this is the opposite (and if so I see the logic of testing the length of all the queue) then I need to find the code that queues commands from the client side. My fault for only concentrating on the grbl buffer without fully tracing through the entire application code.

If you want to make Grbl buffer have more intelligence to decide when to add a delay for EEPROM write time, you could. The problem is, you are starting to add a lot of business logic to Grbl buffer rather than keeping it small and lightweight. The goal with SPJS was to have the smallest amount of code be inside a compiled binary that has a more complicated release structure. Get as much logic higher in the stack into Javascript where the release of new code is as simple as people reloading their browser.