Welcome to the new home of the Cumulus Support forum.

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3044 7 December 2018. See the Wiki for download

Twitter upload problem

Discussion and questions about Cumulus weather station software version 1. This section and its subforums are the main place to get help with Cumulus. Anything which is not specific to the type of weather station goes in here; for anything specific to a type of weather station, please use the appropriate subforum. Use the 'website development' section for any questions relating to creating or running a web site for Cumulus data. Discussion of the stations themselves in these sections is fine.
Post Reply
User avatar
8TX
Posts: 23
Joined: Thu 22 Oct 2009 7:30 pm
Weather Station: WH1080
Operating System: Window 7
Location: Elmstead Market, Essex
Contact:

Twitter upload problem

Post by 8TX » Sun 04 Nov 2018 9:34 pm

I have recently noticed that my software is only updating every 2-3 hrs instead of every 30 minutes which is set in my settings. Is anyone else having this problem?
Marc

freddie
Posts: 330
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Ystrad, Rhondda, Wales
Contact:

Re: Twitter upload problem

Post by freddie » Sun 04 Nov 2018 9:52 pm

Yes. Twitter has recently been enforcing their quota limits of uploads. These are done on a per-user and a per-application basis. As we are all using the same application (Cumulus) then the quota is very soon reached. With an update time of 30 minutes you are more likely to succeed with an upload than me with an update time of 60 minutes - but your upload success rate will probably be similar to mine.
Freddie
Image

User avatar
8TX
Posts: 23
Joined: Thu 22 Oct 2009 7:30 pm
Weather Station: WH1080
Operating System: Window 7
Location: Elmstead Market, Essex
Contact:

Re: Twitter upload problem

Post by 8TX » Sun 04 Nov 2018 10:39 pm

Thanks for the reply,
So really it's only going to get worse then. I think it might be best to leave it as it then

User avatar
ConligWX
Posts: 736
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis Vantage Pro2 Plus
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

Re: Twitter upload problem

Post by ConligWX » Sun 04 Nov 2018 11:09 pm

I upload my data every 60 minutes to twitter without any issue. however if the content is exactly the same as the last tweet, twitter usually does not accept the next tweet.

I find this with tweeting forecasts so I now only upload these about every 4 hours.
Regards Simon

https://www.conligwx.org
https://twitter.com/conligwx
Davis Vantage Pro2 Plus - Meteobrige Nano SD + Saratoga/PWS Templates

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Twitter upload problem

Post by steve » Mon 05 Nov 2018 10:23 am

Twitter keep introducing new ways of making things difficult for third party software developers, including this latest restriction on the number of updates. It is possible to beg them to raise the limit on a per-application basis, so someone might want to take over being the Cumulus developer as far as Twitter is concerned, and jump through the hoops. Here's the email they sent me some time ago:

"In July, we announced that we are making changes to increase accountability on our developer platform and reduce the distribution of spam through our APIs. We are introducing new default app-level rate limits for common POST endpoints on October 22nd.

You are currently using our POST endpoints near or above the new limits. To ensure that your apps will not be impacted when we make these changes on October 22nd, please complete the following steps.

We’ve reached out several times with this information and today’s notice with be our final reminder. If you have not done so already, go to developer.twitter.com/apply and submit a developer account application. Please be as specific and detailed as possible to avoid additional requests for information and ensure a smooth review. Be sure to confirm your email address at the end of the application flow.

While waiting for your developer account application to be reviewed, please complete the remaining steps:

For each app that needs elevated POST access:
Make sure to describe how the app is used in the app details. You can see and edit that field from the Apps page.
Submit an elevation request on our API Policy Support Form:
Select ‘I would like to apply for elevated POST limits.’
Submit the app details and a short description of your use case
We will then review your elevation request along with your developer account application if it has not yet been reviewed. We may need to request additional information, so please monitor the email address associated with your account. We will notify you when your app has been reviewed. You must complete all steps above by October 17th to receive continued elevated POST access.

We know this new process adds extra steps and time to get started with development. We are taking these steps to prevent spam on our platform and are working to determine how best to balance holding developers accountable to our policies with helping developers get started easily.

Regards,
The Twitter Developer team"
Steve

User avatar
ConligWX
Posts: 736
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis Vantage Pro2 Plus
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

Re: Twitter upload problem

Post by ConligWX » Wed 14 Nov 2018 8:11 am

this just affected me today. I post every hour (less than 50 posts a day) - this is just ridiculous.

https://twittercommunity.com/t/new-post ... day/115355
Regards Simon

https://www.conligwx.org
https://twitter.com/conligwx
Davis Vantage Pro2 Plus - Meteobrige Nano SD + Saratoga/PWS Templates

freddie
Posts: 330
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Ystrad, Rhondda, Wales
Contact:

Re: Twitter upload problem

Post by freddie » Wed 14 Nov 2018 9:02 am

steve wrote:
Mon 05 Nov 2018 10:23 am
Twitter keep introducing new ways of making things difficult for third party software developers, including this latest restriction on the number of updates. It is possible to beg them to raise the limit on a per-application basis, so someone might want to take over being the Cumulus developer as far as Twitter is concerned, and jump through the hoops.
I have done this solely to keep our Cumulus tweeting functionality available and usable.
Freddie
Image

User avatar
ConligWX
Posts: 736
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis Vantage Pro2 Plus
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

Re: Twitter upload problem

Post by ConligWX » Wed 14 Nov 2018 11:17 am

freddie wrote:
Wed 14 Nov 2018 9:02 am
I have done this solely to keep our Cumulus tweeting functionality available and usable.
I noticed it happened to Meteobride too. the developer managed to get the app approved by Twitter so allowing users to continue tweeting.

in the meantime I am using Weathercloud to post data to my twitter account.
Regards Simon

https://www.conligwx.org
https://twitter.com/conligwx
Davis Vantage Pro2 Plus - Meteobrige Nano SD + Saratoga/PWS Templates

BigOkie
Posts: 213
Joined: Tue 28 May 2013 1:06 am
Weather Station: Davis VP2
Operating System: Raspian Jessie
Location: Tulsa, OK

Re: Twitter upload problem

Post by BigOkie » Mon 10 Dec 2018 2:40 am

I've been able to get around the issue but it takes a little effort. You'll need to set up as a developer on Twitter and create an app, get your own tokens, and then use a python script with a Twitter library in order to use those tokens. I'm using the RPI3 with Wheezy, so if you have that I can sort of walk you through it. Walking you through getting the App setup on Twitter I can't do, I took me about 30 minutes. Once you have the tokens and keys, however, here are the steps:

You'll need to make sure you have installed or can install Python v3. I don't think v2 will work with it.

You'll need to create a python script that looks like the following (of course replacing the xs with your own keys/tokens):

Code: Select all

#!/usr/bin/env python

from twython import Twython

# Define constants
CONSUMER_KEY = 'xxxxxxxxxxxxxxxxxx'
CONSUMER_SECRET = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
ACCESS_SECRET ='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx''


# Create a copy of the Twython object with all our keys and secrets to allow easy commands.
api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)

text = open('/home/pi/CumulusMX/web/tweet.txt','rb') # Cumulus has been set to create tweet.txt by processing tweetT.txt
api.update_status(status=text)

# api.update_status is now deprecated, but works, and is simpler than the below [untested]
#twitter.update_status(status=text)
Notice in the text = open line there is a path and filename. You don't have to use this filename, however I am. I took the twitter.txt file from the root of the install folder and copied it to the root/web folder and renamed it to tweetT.txt. You'll need to set that filename (path and all) up in the extra files section of the setup and make it generate the filename you want to use. You can replace the above filename from the text = open section with the one you wish to use. I have it set to generate this file every time the standard update generates, which for me is every five minutes.

I would test it first if using Linux/RPI to make sure it works. Issue the command python3 scriptfilename.py. I usually put all my self-made scripts into the /usr/local/bin directory, mainly because it's in the path as part of the OS. Make sure the script is set to executable (usually setting chmod 755 scriptname.py does the trick). If all goes well after python3 scriptname.py, it will move to the next line. If you have a return it likely didn't work. If no return then check the Twitter account you have the tokens/keys set to and see if you have a new Tweet.

To make these work on a schedule, just set up a crontab entry. Mine looks like this one:

Code: Select all

55 0 * * * sleep 5 && /usr/bin/python3 /usr/local/bin/twitter.py /dev/null 2>&1
I put the sleep in there because I have other cronjobs running at every 5 minutes, so putting a delay keeps anything from conflicting. Although they shouldn't. I like to be safe.

Hope this is helpful. If you have any questions I'll try to answer them.

ExperiMentor
Posts: 50
Joined: Tue 24 Nov 2015 11:30 pm
Weather Station: Fine Offset
Operating System: Windows 7; Linux
Location: Switzerland

Re: Twitter upload problem

Post by ExperiMentor » Mon 10 Dec 2018 9:38 pm

BigOkie,

It would have been kind if you'd given me the credit for doing all the hard work and writing that script a few months ago (you've copied it from viewtopic.php?f=4&t=16099#p131775 with only minor changes!)

Cheeky blighter!

Post Reply