LaserWeb Cache Issues

Hello @easytarget,
I just cloned your git and it fails GCode generation. Previous version on which I did my first review still works on the very same DXF.
There are quite few odd things on my setup, maybe due to my lack of knowledge with how npm and everything is supposed to work, along with Chrome cache playing tricks on me (http.server is not running, yet page loads?!). Anyhow here’s what I’ve experienced:

  • everything was updated using apt update; apt upgrade
  • Ran my custom script which does:
  • Cloned fresh new copy from your fork, cloned lw.machines and lw.materials
  • removed package-lock.json
  • npm install --force
  • npm run bundle-prod
  • cd dist then run python3 -m http.server 8001

When I load my DXF, I pick a single color layer, drag it over to the Gcode section, set cut speed, then I click on Generate. At that point in the terminal from which python3 was launched I get a 404 error, it seems its looking for xxyyzz.worker.js I can confirm this file is not present in the git cloned folder.
The browser developer console isn’t much more clear on what is going on.

On a positive note, it does feel snappier on that latest rev with bundle-prod (haven’t checked bundle-dev) but there are errors cropping up in the developer console when I use the mouse scroll wheel to zoom in and out. Performance Display Cache is still needed for my setup (need to restart the web page for the change to take effect when starting with toggle at On?)

You wrote you implemented 3,4 from my original review, but for 3 the grid settings are still with the Machine section. Can’t get to the gcode to generate so I can’t comment on the slider cursor width as the slider is not showing.

I’m sorry you are having issues; What are the contents of the dist folder after you have run npm bundle-dev ?
$ ls -la dist

I just did a full (from scratch) setup using my fork on my desktop box, it all built as expected and I was able to generate gcode, the reason I asked about the dist folder is that you should see the workers as separate entities (.worker.js files, so if they are not present it means that the npm run bundle-dev failed to produce the expected results. But if they /are/ present then you probably have a badly cached index.js which still references the workers by their old filenames.

For reference I’m attaching a log of my test, showing the dist folder before and after the bundle is built. I then ran the resulting frontend and generated some raster merges successfully. You can see the workers being loaded in the final log.

Workers are loaded on-demand, and their filenames are based on a hash of their source; so that you do not run into cache issues when they are updated.

Log
[owen@fc35 ~]$ mkdir LW-TEST
[owen@fc35 ~]$ cd LW-TEST/
[owen@fc35 LW-TEST]$ git clone git@github.com:easytarget-org/LaserWeb4.git
Cloning into 'LaserWeb4'...
X11 forwarding request failed on channel 0
remote: Enumerating objects: 10723, done.
remote: Counting objects: 100% (891/891), done.
remote: Compressing objects: 100% (599/599), done.
remote: Total 10723 (delta 654), reused 495 (delta 292), pack-reused 9832
Receiving objects: 100% (10723/10723), 39.59 MiB | 2.19 MiB/s, done.
Resolving deltas: 100% (7845/7845), done.
[owen@fc35 LW-TEST]$ cd LaserWeb4/

[owen@fc35 LaserWeb4]$ uname -a
Linux fc35.easytarget.org 5.16.16-200.fc35.x86_64 #1 SMP PREEMPT Sat Mar 19 13:52:41 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[owen@fc35 LaserWeb4]$ node --version
v16.13.0

[owen@fc35 LaserWeb4]$ npm --version
8.3.1

[owen@fc35 LaserWeb4]$ rm package-lock.json 
[owen@fc35 LaserWeb4]$ git submodule init
Submodule 'src/data/lw.machines' (https://github.com/LaserWeb/lw.machines.git) registered for path 'src/data/lw.machines'
Submodule 'src/data/lw.materials' (https://github.com/LaserWeb/lw.materials.git) registered for path 'src/data/lw.materials'
[owen@fc35 LaserWeb4]$ git submodule update
Cloning into '/home/owen/LW-TEST/LaserWeb4/src/data/lw.machines'...
Cloning into '/home/owen/LW-TEST/LaserWeb4/src/data/lw.materials'...
remote: Enumerating objects: 125, done.
remote: Counting objects: 100% (125/125), done.
remote: Compressing objects: 100% (77/77), done.
remote: Total 125 (delta 58), reused 115 (delta 48), pack-reused 0
Receiving objects: 100% (125/125), 261.33 KiB | 4.67 MiB/s, done.
Resolving deltas: 100% (58/58), completed with 1 local object.
From https://github.com/LaserWeb/lw.machines
 * branch            db732e001ad58790e99d9a09d783841eb25972af -> FETCH_HEAD
Submodule path 'src/data/lw.machines': checked out 'db732e001ad58790e99d9a09d783841eb25972af'
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 20 (delta 11), reused 18 (delta 9), pack-reused 0
Unpacking objects: 100% (20/20), 3.44 KiB | 135.00 KiB/s, done.
From https://github.com/LaserWeb/lw.materials
 * branch            597c67324ea271d1883dcb33be892a3c8e24ca1b -> FETCH_HEAD
Submodule path 'src/data/lw.materials': checked out '597c67324ea271d1883dcb33be892a3c8e24ca1b'

[owen@fc35 LaserWeb4]$ npm install --force
npm WARN using --force Recommended protections disabled.
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: babel-loader@7.1.5
npm WARN Found: webpack@5.70.0
npm WARN node_modules/webpack
npm WARN   dev webpack@"^5.67.0" from the root project
npm WARN   8 more (css-loader, file-loader, markdown-loader, style-loader, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer webpack@"2 || 3 || 4" from babel-loader@7.1.5
npm WARN node_modules/babel-loader
npm WARN   dev babel-loader@"^7.1.5" from the root project
npm WARN 
npm WARN Conflicting peer dependency: webpack@4.46.0
npm WARN node_modules/webpack
npm WARN   peer webpack@"2 || 3 || 4" from babel-loader@7.1.5
npm WARN   node_modules/babel-loader
npm WARN     dev babel-loader@"^7.1.5" from the root project
npm WARN deprecated core-js@2.6.12: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.

added 558 packages, and audited 559 packages in 2m

55 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm notice 
npm notice New minor version of npm available! 8.3.1 -> 8.5.5
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.5.5
npm notice Run npm install -g npm@8.5.5 to update!
npm notice 

[owen@fc35 LaserWeb4]$ ls -l dist
total 132
-rw-rw-r--. 1 owen owen 123528 Mar 27 10:29 cnctoolpath.svg
-rw-rw-r--. 1 owen owen   1150 Mar 27 10:29 favicon.ico
-rw-rw-r--. 1 owen owen    493 Mar 27 10:29 index.html

[owen@fc35 LaserWeb4]$ npm run bundle-dev

> laserweb@4.1.0 bundle-dev
> webpack --progress --config webpack.dev.js

assets by chunk 1.11 MiB (auxiliary name: main)
  assets by path *.svg 540 KiB 2 assets
  assets by path *.eot 182 KiB 2 assets
  assets by path *.ttf 206 KiB 2 assets
  assets by path *.woff 119 KiB 2 assets
  assets by path *.woff2 93 KiB 2 assets
assets by path *.js 13.3 MiB
  asset index.js 8.92 MiB [emitted] (name: main) 1 related asset
  asset 6b93656f039d2104aa7a.worker.js 1.87 MiB [emitted] [immutable] 1 related asset
  asset 7d8dd8ad9af5da4278eb.worker.js 1.38 MiB [emitted] [immutable] 1 related asset
  asset 6d0de204e96098fc1c20.worker.js 574 KiB [emitted] [immutable] 1 related asset
  asset 624bb0e7e2f635cb6018.worker.js 558 KiB [emitted] [immutable] 1 related asset
  asset 5e366d4941b7c9ae1495.worker.js 14.7 KiB [emitted] [immutable] 1 related asset
runtime modules 27.4 KiB 14 modules
modules by path ../node_modules/ 6.37 MiB
  javascript modules 6.37 MiB 1375 modules
  json modules 3.07 KiB
    ../node_modules/ajv/dist/refs/json-schema-draft-07.json 2.72 KiB [built] [code generated]
    ../node_modules/ajv/dist/refs/data.json 360 bytes [built] [code generated]
modules by path ./ 1.34 MiB
  javascript modules 1.3 MiB 132 modules
  json modules 44.7 KiB
    modules by path ./data/lw.machines/machines/ 13.1 KiB 13 modules
    modules by path ./data/lw.materials/ 31.4 KiB 3 modules
    ./data/macros.json 190 bytes [built] [code generated]
../package.json 3.36 KiB [built] [code generated]
./data/lw.machines/machines/ sync \.json$/ 425 bytes [built] [code generated]
webpack 5.70.0 compiled successfully in 36504 ms

[owen@fc35 LaserWeb4]$ ls -l dist
total 30592
-rw-rw-r--. 1 owen owen   108738 Mar 27 11:01 060b2710bdbbe3dfe48b58d59bd5f1fb.svg
-rw-rw-r--. 1 owen owen   165548 Mar 27 11:01 1e59d2330b4c6deb84b340635ed36249.ttf
-rw-rw-r--. 1 owen owen    77160 Mar 27 11:01 20fd1704ea223900efa9fd4e869efb08.woff2
-rw-rw-r--. 1 owen owen    45404 Mar 27 11:01 4692b9ec53fd5972caa2f2372ae20d16.ttf
-rw-rw-r--. 1 owen owen    20127 Mar 27 11:01 5be1347c682810f199c7f486f40c5974.eot
-rw-rw-r--. 1 owen owen    15005 Mar 27 11:01 5e366d4941b7c9ae1495.worker.js
-rw-rw-r--. 1 owen owen    23721 Mar 27 11:01 5e366d4941b7c9ae1495.worker.js.map
-rw-rw-r--. 1 owen owen   571338 Mar 27 11:01 624bb0e7e2f635cb6018.worker.js
-rw-rw-r--. 1 owen owen   579574 Mar 27 11:01 624bb0e7e2f635cb6018.worker.js.map
-rw-rw-r--. 1 owen owen  1960421 Mar 27 11:01 6b93656f039d2104aa7a.worker.js
-rw-rw-r--. 1 owen owen  2363733 Mar 27 11:01 6b93656f039d2104aa7a.worker.js.map
-rw-rw-r--. 1 owen owen   587912 Mar 27 11:01 6d0de204e96098fc1c20.worker.js
-rw-rw-r--. 1 owen owen   742258 Mar 27 11:01 6d0de204e96098fc1c20.worker.js.map
-rw-rw-r--. 1 owen owen  1448933 Mar 27 11:01 7d8dd8ad9af5da4278eb.worker.js
-rw-rw-r--. 1 owen owen  1886428 Mar 27 11:01 7d8dd8ad9af5da4278eb.worker.js.map
-rw-rw-r--. 1 owen owen    23424 Mar 27 11:01 82b1212e45a2bc35dd731913b27ad813.woff
-rw-rw-r--. 1 owen owen   165742 Mar 27 11:01 8b43027f47b20503057dfbbaa9401fef.eot
-rw-rw-r--. 1 owen owen    18028 Mar 27 11:01 be810be3a3e14c682a257d6eff341fe4.woff2
-rw-rw-r--. 1 owen owen   444379 Mar 27 11:01 c1e38fd9e0e74ba58f7a2b77ef29fdd3.svg
-rw-rw-r--. 1 owen owen   123528 Mar 27 10:29 cnctoolpath.svg
-rw-rw-r--. 1 owen owen    98024 Mar 27 11:01 f691f37e57f04c152e2315ab7dbad881.woff
-rw-rw-r--. 1 owen owen     1150 Mar 27 10:29 favicon.ico
-rw-rw-r--. 1 owen owen      493 Mar 27 10:29 index.html
-rw-rw-r--. 1 owen owen  9348381 Mar 27 11:01 index.js
-rw-rw-r--. 1 owen owen 10456540 Mar 27 11:01 index.js.map

[owen@fc35 LaserWeb4]$ cd dist
[owen@fc35 dist]$ python3 -m http.server 8000
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [27/Mar/2022 11:05:56] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [27/Mar/2022 11:05:56] "GET /index.js HTTP/1.1" 200 -
127.0.0.1 - - [27/Mar/2022 11:05:57] "GET /cnctoolpath.svg HTTP/1.1" 200 -
127.0.0.1 - - [27/Mar/2022 11:05:58] "GET /20fd1704ea223900efa9fd4e869efb08.woff2 HTTP/1.1" 200 -
127.0.0.1 - - [27/Mar/2022 11:06:34] "GET /6d0de204e96098fc1c20.worker.js HTTP/1.1" 200 -
127.0.0.1 - - [27/Mar/2022 11:06:36] "GET /624bb0e7e2f635cb6018.worker.js HTTP/1.1" 200 -

Edit: I strongly suspect a cache problem… since the grid settings are definitely moved from machine to applications settings in the latest version…

2 Likes

Hi,
It must indeed be something related to the cache. I can start either the latest git clone (version 4.1.0) or the one from February (version 4.0.999) and Chrome still shows me whatever was last from cache. If I open up an Incognito tab in Chrome it is not affected like normal tabs. However the machine and application stored parameters are not available in the Incognito tab.

Funny thing now, I have started up the 4.0.999 server but the 4.1.0 is in the cache, I can run and generate GCode. Even asking Chrome to reload the page doesn’t fix it. I’ve seen options to disable the cache altogether in the development console, but for me it doesn’t even work, cached version is still loading.

In order for the correct version to load, I had to clear the browsing data but only Cached images and files with an All time range in Chrome.

Not sure if I’m the only one affected by this, but maybe we should consider cache busting. That link is kinda old (2014) but anyway, food for thoughts.

Now that things are back on track on my side, I can confirm items 3 and 4 on my original punch list are now in good order. I also like the fact that the server connects up to the machine as soon as you open the Comms panel.

I’m going to suggest a patch for lw.comm-server MPG soon. After that I’ll be able to dig on the force-closed/rogue line reported in another thread.

In Chrome Open the developer tools (rt-click->inspect) and leave them open, mimnimize if distracting. Then go to the refresh page icon in the URL bar and long press it. Select ‘Clear Cache and Hard Reload’, this is the safest way to load a page when developing.

These are browser caching issues; and I don’t really see them; I test a lot on Chrome, Chromium and FFox without issue… I dont have time to go through all the discussion here, chrome is deciding how to cache the page based on heuristics… the way to override these is to set the correct headers when serving the pages.

Create a python program called, say, python-http-server-no-cache.py

#!/usr/bin/env python
try:
    from http import server # Python 3
except ImportError:
    import SimpleHTTPServer as server # Python 2

class MyHTTPRequestHandler(server.SimpleHTTPRequestHandler):
    def end_headers(self):
        self.send_my_headers()
        server.SimpleHTTPRequestHandler.end_headers(self)

    def send_my_headers(self):
        self.send_header("Cache-Control", "no-cache, no-store, must-revalidate")
        self.send_header("Pragma", "no-cache")
        self.send_header("Expires", "0")


if __name__ == '__main__':
    server.test(HandlerClass=MyHTTPRequestHandler)

Save in the project root, then run from the dist folder.

[owen@fc35 LaserWeb4]$ cd dist
[owen@fc35 dist]$ python ../python-http-server-with-cache.py 
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [30/Mar/2022 17:40:52] "GET / HTTP/1.1" 200 -

This tells the browser not to cache anything

The worker js files already use a content-hash as a filename, they naturally cache bust. Extending this to the main .js file is less simple, but vaguely in my roadmap.

PS; I’ve fixed the irritating console stuff on mouse zoom. Look for me to push that soon.

1 Like

Hi,
I tried the python script, went back and forth between the two commits and unfortunately it doesn’t work on my Debian install with chromium 99.0.4844.84. The trick with Empty Cache and Hard Reload does however work.

Anyway I feel this is more a development annoyance as a normal user wouldn’t be affected except during version upgrades. I suggest to add a warning in the readme to clear the browser cache when changing versions.

I just moved this OT discussion to a separate thread.
Also edited my post above to un-hide the python script and I have edited that script to send the ‘slam-dunk’ do NOT cache header set.
If that doesnt work; then your browser has a config / extension issue.

However; for web development the developer console + hard reload trick is the correct and normal practice.

A little extra note;

npm run start-dev

Will start webpack in CI mode, serving to a URL shown in the output.

  • it should also start a new browser window or tab to the served site; this depends on your UI etc so ymmv, use the URL in the output if this doesn’t happen.

This is a nice development environment; if you save/update any source file the project will be rebuilt and redeployed (fast), and the browser will auto-reload once it is built, or show you the errors if it fails.

1 Like

The revised Python script does the job on my setup.
Regarding the npm run start-dev I was using it a while back on the original dev-es6 branch.