Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

import Cumulus file.php doing bad, worse and worse

Other discussion about creating web sites for Cumulus that doesn't have a specific subforum

Moderator: daj

Post Reply
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

import Cumulus file.php doing bad, worse and worse

Post by mm23 »

So, in the beginning everything works well and importCumulus php file worked well. By sometime, I have trouble opening realtime.txt file from the server. Sometimes it opens the entire file, sometimes just part of the file and sometimes just the last line.
Almost everything is working well :D , including a cron job every minute, however, missing data in the realtime table but cron job is executed well. People from web hosting say that everything works well, but I'm missing data in realtime table in the database and on the graph on the website.

Any help, please........

http://www.vrijemeusibeniku.com/weather/graf1.php
You do not have the required permissions to view the files attached to this post.
Image
User avatar
mcrossley
Posts: 12697
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: import Cumulus file.php doing bad, worse and worse

Post by mcrossley »

Do you have the ftp 'rename' option enabled in Cumulus?
User avatar
rogerthn
Posts: 489
Joined: Thu 11 Apr 2013 6:31 pm
Weather Station: Ecowitt GW1000/GW1003
Operating System: Raspberry Pi OS bullseye aarch64
Location: Trollhättan Sweden
Contact:

Re: import Cumulus file.php doing bad, worse and worse

Post by rogerthn »

mm23 wrote:... sometime, I have trouble opening realtime.txt file from the server. Sometimes it opens the entire file, sometimes just part of the file and sometimes just the last line....
I've seen something similar (not Cumulus) and the reason was that sometimes transfer of the file wasn't complete when processing started and sometimes a new file was created before processing was complete.
Image
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

Re: import Cumulus file.php doing bad, worse and worse

Post by mm23 »

mcrossley wrote:Do you have the ftp 'rename' option enabled in Cumulus?
No, I have not
Image
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

Re: import Cumulus file.php doing bad, worse and worse

Post by mm23 »

rogerthn wrote:
mm23 wrote:... sometime, I have trouble opening realtime.txt file from the server. Sometimes it opens the entire file, sometimes just part of the file and sometimes just the last line....
I've seen something similar (not Cumulus) and the reason was that sometimes transfer of the file wasn't complete when processing started and sometimes a new file was created before processing was complete.
Hm, I am not sure.. In the beginning everything was ok .... now, there is no rule ..... sometimes works great, sometimes not and since last night does not work at all.....I mean web pages...
Image
User avatar
mcrossley
Posts: 12697
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: import Cumulus file.php doing bad, worse and worse

Post by mcrossley »

mm23 wrote:
mcrossley wrote:Do you have the ftp 'rename' option enabled in Cumulus?
No, I have not
If you enable it and your web site supports it OK, then that may fix the problem of the truncated files for you.
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

Re: import Cumulus file.php doing bad, worse and worse

Post by mm23 »

OK Mark, I'll try it. ....
Today afternoon, I turned off the cron job on the server and turned on again http comand from Toolbox and everything is start to "working" again. The only problem is that the http command activates on every hour and cron job every minute..... The question is why the "system" does not work when it activates by a cron job, and it works when it is activated by http command from the Toolbox ....?
Image
User avatar
rogerthn
Posts: 489
Joined: Thu 11 Apr 2013 6:31 pm
Weather Station: Ecowitt GW1000/GW1003
Operating System: Raspberry Pi OS bullseye aarch64
Location: Trollhättan Sweden
Contact:

Re: import Cumulus file.php doing bad, worse and worse

Post by rogerthn »

mm23 wrote:... The question is why the "system" does not work when it activates by a cron job, and it works when it is activated by http command from the Toolbox ....?
What does The crontab line look like?
Maybe you can try adding something like >> /tmp/my.log 2>&1
Please note the >> i.e. append i.e. the file will get bigger and bigger ;-)
Image
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

Re: import Cumulus file.php doing bad, worse and worse

Post by mm23 »

After the http command from Toolbox worked well all night, I decided to turn back to old settings (re-activate the cron job on the server - one minute interval) and now everything is ok again. I did not activate the ftp rename option (for now) as Mark suggested - thanks Mark. What happened still do not know.
Php command is ......php -q /path to/importcumulusfile.php type=realtime key=***** table=***** file=yourrealtimefile retainVal=2 retainUnit=day......and as you see the table size is limited on two days

Anybody have any suggestion what might be causing my problem because I doubt that my problem still has not been solved by long-term?

Thanks
Image
User avatar
mcrossley
Posts: 12697
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: import Cumulus file.php doing bad, worse and worse

Post by mcrossley »

It just a clash between the FTP upload and the cron job reading the file. Using the rename should eliminate the small time period when there is a partial file available to read.

Edit: phone tryping
Last edited by mcrossley on Sat 17 May 2014 4:56 pm, edited 1 time in total.
mm23
Posts: 152
Joined: Mon 03 Feb 2014 12:22 pm
Weather Station: ws 2350
Operating System: windows xp sp3
Location: Europe

Re: import Cumulus file.php doing bad, worse and worse

Post by mm23 »

OK, Mark thanks. I will do so and see what will happen......
Image
Post Reply