Problem with realtime.txt

[jank]

Hi
today I decided to recreate my CumulusMX Raspebrrypi since it was messed up over the time.
I used the RaspberryPi Image provided from here and most of the stuff is working fine.
However I have Problems with the realtime.txt

1. In RaspberryPi Image the path for the both files is a symlink and points to
wxnow.txt -> /run/cumulusmx/wxnow.txt
realtime.txt -> /run/cumulusmx/realtime.txt

for any reason, the folder /run/cumulusmx is not being created automatically so I am getting the following error
FTP[13]: Error uploading realtime.txt to /www/wetter2/realtime.txt : Could not find file "/opt/CumulusMX/realtime.txt"

After creating the folder /run/cumulusmx/ manually suddenly both files are being created but realtime.txt is still empty

root@cumulusmx:/opt/CumulusMX# ls -al /run/cumulusmx/
total 4
drwxr-xr-x 2 root root 80 Apr 25 17:05 .
drwxr-xr-x 27 root root 940 Apr 25 17:04 ..
-rw-r--r-- 1 root root 0 Apr 25 17:55 realtime.txt
-rw-r--r-- 1 root root 55 Apr 25 17:55 wxnow.txt


Any Idea whats wrong with the realtime.txt ? ...and why the /run/cumulusmx folder is not being created ?
btw: realtimegauges.txt is properly filled with data
thanks for any advice
Jan

Update: In the MXdigs Logfile I can find the following error:

2022-04-25 18:04:37.357 RealtimeLocalCopy[0]: Error copying [realtime.txt] to [realtime.txt. Error = Sharing violation on path /opt/CumulusMX/realtime.txt

[jlmr731]

the directory must be there as the program will not create it for you, but now you have created it so all should be good.
Dont know much on the setup of the image for pi but looks like the program is in /opt/CumulusMX so realtime should be there, if not have you tried deleting the file (realtime.txt) and restarted cumulus to see what happens also not sure why it would be a symbolic link to /run/cumumusmx

[sfws]

Any Idea whats wrong with the realtime.txt ? ...and why the /run/cumulusmx folder is not being created ?
btw: realtimegauges.txt is properly filled with data
thanks for any advice
Jan

Update: In the MXdigs Logfile I can find the following error:

2022-04-25 18:04:37.357 RealtimeLocalCopy[0]: Error copying [realtime.txt] to [realtime.txt. Error = Sharing violation on path /opt/CumulusMX/realtime.txt

That error message tells you what is wrong, you are getting a sharing violation, either you are asking MX to generate a new file before it has finished creating the previous one, by using too short a realtime interval or something else is reading that file, rather than the copy MX is trying to make, stopping the file being re-created by MX.

I don't use the image, but I can read the documentation about it at https://cumuluswiki.org/a/Raspberry_Pi_Image, that tells me that the symbolic links exist to force each temporary file to be created in RAM reducing wearing out of the micro-SDHC card.
That explains why the image does not have a physical folder called "/run/cumulusmx", it exists (or should do) in the memory chip on your computer, not on the card folder structure where you were looking.
EDIT - as Mark says in subsequent post, "/run" is in file system on disc, it should hold symbolic links for the temporary files, he had a mistake in his image, and the symbolic links should point to files in the memory, each of which in his 3.16.0 image will be the normal temporary file name prefixed by "cumulusmx-". Looks like the error in the particular image you happened to pick caused confusion to both of us! (The Wiki page needs to explain this better).

After creating the folder /run/cumulusmx/ manually suddenly both files are being created but realtime.txt is still empty

You probably need to start again with a fresh image, as you messed everything up by creating that folder.

Let me quickly recap how Cumulus works, it processes a "realtime.txt" template, either the internal default, or your own template, (see https://cumuluswiki.org/a/Realtime.txt).
Regardless it creates the processed file in RAM (or it did before you messed up your system) at "/run/cumulusmx/realtime.txt", and then (on your settings) copies the file to the folder declared in (MX interface) Internet settings --> Web/FTP Site --> Local copy destination folder path:
and hopefully you entered there a sensible path, outside the folder "/opt/CumulusMX" structure.
Whatever you have accessing "realtime.txt" needs to access it at this last mentioned place, at times other than when MX is recreating the file, and not to apply any file lock, a normal read does not need to apply a lock.
I don't know how what interval you have set for real-time, but if it is too short, one read/write might be getting tangled with the next, so set it to a longer interval to begin with, and reduce it gradually, checking there are no problems.
Are you sure a new empty copy of realtime.txt is being created at each update? It may be empty because the sharing violation stopped MX copying data to it, or it may be empty if you have not set "Generate" in the settings, or it may be empty if you have defined a template to use with an error in that template.

Mark does not say, in the Wiki page he created about the image, how he creates the "/run/cumulusmx" folder, nor does he say at viewtopic.php?p=150310&hilit=Raspberry+image#p150310, so I don't know if he uses the technique at https://cumuluswiki.org/a/MX_on_Linux#M ... ge_storage or the technique described at viewtopic.php?p=162155#p162155. Therefore, I cannot explain how you can put right the folder assignment that you have mucked up.

Anyway trying to get back to the right set up now you have altered stuff is probably unwise.Go back and start with a new image, and as I say check your realtime interval settings, and check whatever you have that reads the realtime file.

[mcrossley]

The /run folder is created by the OS at start-up.

The /run/cumulusmx subfolder should also be re-created on start-up. I'll be updating the pi image for the next release, I'll take a look at this issue then.

[mcrossley]

I just checked, its a mistake in the 3.15.0 image, it should not use a sub-folder of /run, it should be creating the temp files with a "cumulusmx-" prefix.

I'm just creating the 3.16.0 image, so I'll fix that.

[jank]

Thank you jlmr731 , sfws (for the detailed description) and Mark, for the explanation.

so finally it was a small mistake in the image, since there should not be a subfolder /run/cumulusmx .....(Fixed by Mark with a fresh Image)
however, meanwhile I installed a fresh raspbian image and installed cumulusmx via ZIP File....unfortunately without the advantage of all the RAM files......anyway, so far, I never had a problem with corrupted SD Cards ....I am using CumulusMX on a RaspberryPi with SD Card since years without ever having problems.....
I stuck with a very old version of CumulusMX since months (Or years since I did not had the time to update CumulusMX in the past for any reason) but now I am on the most current version and I saw, that a lot of work was done by Mark in the past months.....
This is the CPU Load of my RaspberryPi Zero with the old version of CumulusMX (Cumulus MX v.3.10.2 build 3117) and the new Version (Cumulus MX v3.16.0 build 3182) :


The much higher CPU Load (starting on thursday) is being caused by mono......but I assume, this is normal since a lot more functionallity was implemented in CumulusMX in the past months? So I also assume that a old RaspberryPi Zero W might be to weak from its performance ?
Am I right regarding the higher CPU Load of Mono.
The versions of Mono on the old CumulusMX and the new CumulusMX versions are the same: Mono JIT compiler version 6.12.0.122

Thank you in advance for you suggestions