X10 Community Forum

📱🖥️PiX10Hub => 🏥Help and Trouble Shooting => Topic started by: shtckt on June 16, 2020, 10:26:29 PM

Title: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 16, 2020, 10:26:29 PM
I had left my system untouched, and non-upgraded, from early January. Got frisky after seeing a succession of updates. Updated with stable.13 yesterday afternoon and now WeatherAlerter died. Well, sort of. The time updates but the weather stats are frozen.

What is says when I load up the options is "CR,178,23: Unreachable code detected". Alerter will not respond to a restart of program, HG, or a Pi reboot.

So I figured I would just go for the All Pi version. Installed well. Then realized I did not know how to install both WeatherAlerter and DuckDNS from  Tuicemen's forum. I tried the obvious way using HG program/module install. It poked WeatherAlerter into a generic program listing instead of the weather category. I did not even try DuckDNS.

So put that SD creation on hold and went back to the SD with the crippled WeatherAlerter as at least all my X10, schedules, etc all ran just fine. That was the first Buster image. I backed it up and then tried to restore the settings made before I allowed the HG update. In my cockiness to use the HG update, I did not make an SD backup. In fact, I had already taken my Jan backup SD to create the All Pi SD. Just shoot me.

Would love the WeatherAlerter but it will not deter me from using my X10PiHub without it.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 16, 2020, 10:33:27 PM
That error code was in the widget's "Compiler.Warning".

When I went back to get that above widget setting or notice, I discovered HG was now stable.14.
So I updated. Weather Alerter still had the same symptoms and Compiler Warning. It is like it quit getting info from service feed.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: Tuicemen on June 17, 2020, 07:28:36 AM
Im not seeing an error on my phone and weather info seems to update in weather alerter (other then humidity)  Gene did fix a bug with Openweather I belive that was introduced with stable 13 however never added the humidity fetch code. I'll try to look into this a bit more with my laptop this weekend.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: Tuicemen on June 17, 2020, 07:43:32 AM
It should be noted Weathealerter and Duck DNS programs can both be found on this forum here http://forums.x10.com/index.php?board=172.0 the only difference in the install process is you have to rename the files here on my forum you don't  have to.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: soxfan1966 on June 17, 2020, 08:10:56 AM
I am running HG release 1.3 stable 14 (updated yesterday) and also have Weather Alerter.  I am not getting any errors in that program and it is updating as well.

The Compiler Warning mentioned can also be cleared (go to Configure / Groups and expand the group your Weather Alerter widget is in, then click the hamburger icon - the 3 vertical lines - for the widget.  In the list of Parameters, click into the box for Compiler.Warning and then click the Delete footer button, then Save).  Click Home to get back to the Main HG display.  You could also navigate to Comfigure / Programs, and then edit Weather Alerter and then Compile it to see if you get any errors there).
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 17, 2020, 11:24:52 AM
Thanks all for the advice.

I reset/deleted the error. Restarted program. It came back with the same error.

I got the 1006-Weather_Alerter, and the tuicemen_tools_info files from Tuicemen's website. After removing the old program, I imported the downloaded version. Then I imported the widget. The program installed. I initiated a start and it sent me the same data from two or three days ago but with the present date/time.

When I attempted to add the widget to the Dashboard, it was not in the tuicemen_tools_info section. It could not even be found to be bound. I remember on my previous build of the image, WeatherAlerter was in its own named section that had a tornado icon. That section does not exist after the import. I looked in various sections and only found where I could bind the WeatherAlerter widget was in the homegenie_generic_unknown folder. Tried that but stopped when the widget image looked nothing like the original. If I use Windows Explorer and open the tuicemen_tools_info zip file, I can see the tornado icon png.

Since both Status and DuckDNS work flawlessly, I am hesitant to delete the  tuicemen_tools_info section and reinstall. I seem to have crippled this HomeGenie installation and may well live without the WeatherAlerter.

Looks like I will return to getting the All Pi version of image from Tuicemen to work with the tuicemen_tools_info, etc, but it seems whatever I am doing arrives at the same conclusion as what I did with this image installation that has worked since January.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: petera on June 17, 2020, 11:51:42 AM
Not getting involved in a 3rd party install of HG but did you at any stage take an SD card image of the SD card you are using that you could possibly restore at this stage that would get you back to your January setup. Even something close to that date that you could incrementally update.

Firstly there was a major release of Raspbian Buster in February and a number of releases of HG and assuming you’re using HA Bridge possibly an update of that to contend with. I’m assuming you have an update schedule for all these components in place.

What you could do at this stage is do a factory settings reset of HG and restore the last know good HG configuration backup and take things from there.

Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 18, 2020, 11:24:57 AM
By the way, I am using a ZeroW with a 4 port Ethernet USB dongle to cable to a X10 CM15. I had the standard Pi power supply connected to a dedicated APC 550 Backup UPS. Last week, to try to eliminate the PS as a problem, I swapped out the PS with a third party 2.5 amp version. I let it run for this last week and I saw no difference in the PiHub performance so assume the old PS was not contributing to my HG problems.

While I had reused my backup SD to burn an All Pi image, what I did have was a set of IMG files I had made from late Dec 2019 thru 9 Jan 2020. Was making incremental changes in HG and then making a backup before trying something new. Then we went on holiday and the next backup of anything other than configbackups was June 2020. So I burned an SD with with the 9 Jan 2020 IMG.

This was the version I had PiHub running for months until I decided this last week to grab the string of June 2020 HomeGenie updates. But I knew there were problems with this 9 Jan version as it was the one that ate up free memory like it was the cookie monster.

Every day on holiday I would remotely fire up HG on my laptop to focus on the Status widget's MemFree. If it got below 80Mb, I would run Tuicemen's Memory Cleaner and it would eventually show over 200Mb. What I did not know, and still do not know, is why it was eating memory. I would remotely run PuTTY and list with "top -o %MEM". Would scroll down to the items that listed zero memory use, grab the list of those with usage, and enter them into a spreadsheet. Memory total would never show over 35%-40% including Terminal when using PuTTY. But in the header after running tops it did show the same free memory amount that the Status widget showed. After I came home from holiday, I let the system run down on free memory. That last I remember seeing was something south of 15Mb because the next time I tried to check I could not open HG in a browser. It was froze. I could run PuTTY and I used it to sudo shutdown the Pi. In my panic, I did not run "top". Went directly to shutdown, checked the SD on my laptop. I could read the SD. Put it back in the Pi and fired it up. It ran just find with initially over 230Mb free memory.

After I upgraded to HG stable.13...from stable.4, that was when things really went downhill. What I know as of this morning was sometime after the HG update, OpenWeatherMap would run and then die. That is why WeatherAlerter would update date/time but none of the meteorological information. If I restarted OpenWeatherMap, it would update the WeatherAlerter widget for one single cycle and then stop.

Last evening, I burned and SD with the 9 Jan 2020 IMG. WeatherAlerter and OpenWeatherMap played nice together. But I still had the memory being eaten in chunks. I let it run overnight. This morning it showed less than 42Mb free. But everything played nice.

That 9 Jan 2020 IMG was the last one with my pre-holiday final X10 Schedule and other settings. Hated to see that one go away but decided to burn an SD with the 2 Jan 2020 IMG, swapped out the 9 Jan SD, and painstakingly updated the Schedule and settings. Free Memory does diminish some but eventually will rise. Had not seen that happen in months.
This 2 Jan image is Buster version 10 (has to be release 2019-09-26) and HG stable.4 or at least is says stable.5 thru stable.14 are available for updates.

Everything in my programs and X10 Modules that are scheduled to start/stop each morning has been accomplished. Everything is still playing nice with each other. Have taken all kinds of readings via PuTTY and downloaded the logs. I am going to shutdown the PiHub in a few minutes and make an image backup onto a backup SD card. I will store the SD that is running now and will run on the backup SD until I see it is functional and all the setting and programs run as expected.

If I make no changes in the next 24 hours, I will start the process of updating Pi OS and HG. I presume that is the proper order. Pi OS last update shows on Wiki as still Buster version 10 but is release 2020-02-13.

Thanks for the advice Petera about just going back in time and finding a known good configuration. I have temporarily abandoned preparing the AllPi image as there is something that I am not doing correctly when importing TuicemenTools programs and widgets that includes WeatherAlerter and DuckDNS. Neither of the widgets are showing correctly during the configuration stage even after restarting HG or rebooting the PiHub. WeatherAlerter's widget keeps getting dumped into homegenie/generic/unknown.  DuckDNS shows a bunch of miscellaneous text written across the widget. I am sure there is something I am not doing as these items were part of previous Tuicemen images which worked quite well. But that is a project for a later time. I need to take a break and let this old Buster and HG run.

Last check showed over 200Mb of FreeMemory. What a joy to behold.

Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: petera on June 18, 2020, 12:56:34 PM
That was an exhausting write up I imagine  :)%

Ok back to a familiar subject about the Pi Zero W and a 4 port hub.What else is running off that hub besides your X10 controller. If nothing else is, remove it and use a single USB to USB C adaptor. Those 4 port hubs cause mayhem with the Pi Zero W.

Secondly it sounds like you have quite a few other memory gobblers besides HG chewing up available memory.Are they really needed.The simpler your setup is the less likely it is to crash.There should never be a need to restart the HG server unless its purely for maintenance purposes no matter what anyone says.Memory leaks caused by certain programs can cause this unwanted side effect that can eventually crash your system.This appears to be the case in your situation.Rising memory usage resolved by a server reboot.Not good.

Thirdly, if any of these other 3rd party products are doing a lot of writing to your SD card your SD card will eventually fail. Again being 3rd party I can't and won't comment other than to say you sound like you have your Pi Zero W under heavy load constantly.Unless you are one of these log freaks you can disable logging in HG unless you need it on for debug purposes.

Not entirely sure why you are using DuchDNS.Maybe because it's free.If it's because it's free remember that Ducksdns does nothing for security. It simply takes your hard to remember public IP address and translates it to an easier to remember domain name.

The SSL cert with let’s encrypt let’s you encrypt your traffic while in transit. This means any passwords or other private information isn’t broadcast for all to see.

Be aware if you are using this method you likely have an open port on your router to allow external access. That is an open door to the world. People and bots will check that door regularly. Be sure you have a very strong password and it is highly recommended to set up two factor authentication.

Now it's a case of peeling back as much of the load as possible from the PI Zero W or looking at one of the full size Pi boards to run your system on and do remember that 1)Backup your system,then back it up again, 2) Rising memory usage is usually a sign of trouble ahead and 3) If you ignore 1) and 2) be sure to be prepared for long days sorting out the consequences. I really do recommend having a second SD card with your system image written to it located beside your Pi. It's then just a simple swap instead of the alternative :'
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 18, 2020, 01:52:07 PM
Agree with all said.

I run DuckDNS on the Pi just to get another view of the WAN address. Being free did not hurt. I have a DDNS service thru my router's manufacturer.

I use two-factor authorization on the Pi.

I have stayed with the 4-port USB Ethernet adapter as I have purchased adapters only to find the throat of the USB Micro portion is not long enough to penetrate adequately the port with the Pi case installed. They make some with a longer throat but they want a king's ransom. Will keep looking or may end up building one.

I have disabled or removed all the HG programs not used. I am not aware of what can be trimmed from the OS.

I have 20 modules of X10, the DuckDNS, WeatherAlerter, and Status programs/widgets. The WeatherAlerter, from what I can determine, requires WeatherMap. I have and used Tuicemen's tools for memory freeing, disk freeing, date/time reset, Pi reboot, and Pi shutdown.

Seasonally I enable 6 other X10 modules for outside and inside receptacles. They are turned off via HG otherwise.

I have 30 lines of scheduled events that include SolarTimes functions a well as set times.

I have changed DuckDNS to function hourly instead of every 5 minutes. After all, it is a convenient backup.

Since the Pi memory situation seems to have been resolved, I have enabled log and events via HG.

I will look at the SSL certification. Thanks for the tip.

I just checked and the Pi has over 240Mb free memory. That is the typically the amount seen right after a boot. I have checked every 30 minutes or so today and the number will go to the high 100Mb but it bounces back above 220Mb or higher. I feel whatever was robbing the free memory, is diminished or gone.

I do keep two via SD cards with the present system. But somewhere in the last 3 months I made them identical again not realizing the master had some faults. I have two more SD cards that I am stalled on the All Pi upgrade. I keep a series of IMGs along the way that are on my cloud account and not on SD.

Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 18, 2020, 02:05:02 PM
I am juggling too many balls in the air at one time. I have a 4-port USB dongle. It has nothing to do with Ethernet. I use wireless Ethernet to communicate with the Pi.

Free memory is now 229Mb. Decided to give it 24 hours of running before I pull the SD, put in the backup SD, run it for 24 hours, and start updating the OS and HG.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: shtckt on June 18, 2020, 03:48:43 PM
Was looking for something else and another thread caught my eye...

http://forums.x10.com/index.php?topic=31311.msg185620#msg185620 (http://forums.x10.com/index.php?topic=31311.msg185620#msg185620)

Seems the WeatherAlerter problem I had was probably the OW certificate. Only the date/time updated but no other weather data. I had previously searched for some discussion about the problem but can see my search criteria was not in the article. In my case, I also noticed that  OpenWeatherMap program turned off after the HG...stable.13 update. Manual efforts to restart would update the date/time but not the weather data.

I had an API with WeatherUnderground long ago that would send data to a Windows Gadget. The service would get dropped by WU, get brought back to life for a while, then would periodically get dropped. I finally gave up.

I did get another Gadget to use. Is called My Weather. Copyrighted to a Pat Possible. The author has a website [ https://00ff00.de/],and email [hallo@00ff00.de] to post problems and ideas. This page,  https://00ff00.de/?p=228, has more info on the Gadget.  Achtung, Heir Tuicemen, it might have an opening to another free API or service.  Oh, on second thought, hold grilling the bratwurst, I just searched through the 5Mb zipfile used for installation for clues. The author has a folder on OpenWeather.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: petera on June 18, 2020, 04:58:00 PM
Was looking for something else and another thread caught my eye...

http://forums.x10.com/index.php?topic=31311.msg185620#msg185620 (http://forums.x10.com/index.php?topic=31311.msg185620#msg185620)

Seems the WeatherAlerter problem I had was probably the OW certificate. Only the date/time updated but no other weather data. I had previously searched for some discussion about the problem but can see my search criteria was not in the article. In my case, I also noticed that  OpenWeatherMap program turned off after the HG...stable.13 update. Manual efforts to restart would update the date/time but not the weather data.


I had an API with WeatherUnderground long ago that would send data to a Windows Gadget. The service would get dropped by WU, get brought back to life for a while, then would periodically get dropped. I finally gave up.

I did get another Gadget to use. Is called My Weather. Copyrighted to a Pat Possible. The author has a website [ https://00ff00.de/],and email [hallo@00ff00.de] to post problems and ideas. This page,  https://00ff00.de/?p=228, has more info on the Gadget.  Achtung, Heir Tuicemen, it might have an opening to another free API or service.  Oh, on second thought, hold grilling the bratwurst, I just searched through the 5Mb zipfile used for installation for clues. The author has a folder on OpenWeather.

How long are you using HG.What else have you got running on that SD card.You say you are using a 4 port ethernet adaptor.Surely you mean 4 port USB hub.If so is it powered.Is anything else attached to the 4 port hub besides your X10 controller.All these factors will effect performance.As you don't mention HA Bridge do I assume you don't use it.

I've no idea what Weather Alerter is or what it does.I assume it somehow alerts you via notification in the event of weather condition changes.This is something you can script yourself with some simple Wizard automation scripts.

Yes Open Weather security certs expired but were replaced within 24 hours.This should not have brought you system to a standstill. Yes HG has had a number of updates since January but again from what I can figure out from your setup none that would have crippled your system.

Running 3rd party scripts in HG can have adverse effects particularly if they are not maintained to reflect changes in the main system.Updates are best done incrementally.Just because you saw a V13 update it doesn't mean you have to take it There's a manual update feature available in the HG web UI that allows you to install whatever version you like as long as it's equal to or greater than the version you currently have installed.

A simple command from a PuTTY session in Windows like, for instance, to get version 9 would look like wget https://github.com/genielabs/HomeGenie/releases/download/v1.3-stable.9/homegenie_1.3-stable.9_all.deb and so on.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: petera on June 18, 2020, 06:20:05 PM
I've mentioned this on the forum before but just to finish with my input on this. I recommend the following setup with a Pi Zero W running HG

Pi Zero W (obviously)
Power supply 5v 2.5ah preferably
X10 controller (CM11,CM15 or the CM19 which will draw power from the Pi Zero W
USB to mini USB OTG cable https://shop.dexterindustries.com/otg/

HG server only will run fine on the above setup but any other server or additional software would require a full size Raspberry Pi. While the Zero might appear to run fine fully loaded it will affect the performance of HG.The OTG USB cable has electronics already built in and its own drivers to operate it so connecting to a 4 port USB hub causes firmware conflicts .
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: petera on June 20, 2020, 11:29:27 AM
Was looking for something else and another thread caught my eye...

http://forums.x10.com/index.php?topic=31311.msg185620#msg185620 (http://forums.x10.com/index.php?topic=31311.msg185620#msg185620)

Seems the WeatherAlerter problem I had was probably the OW certificate. Only the date/time updated but no other weather data. I had previously searched for some discussion about the problem but can see my search criteria was not in the article. In my case, I also noticed that  OpenWeatherMap program turned off after the HG...stable.13 update. Manual efforts to restart would update the date/time but not the weather data.

I had an API with WeatherUnderground long ago that would send data to a Windows Gadget. The service would get dropped by WU, get brought back to life for a while, then would periodically get dropped. I finally gave up.

I did get another Gadget to use. Is called My Weather. Copyrighted to a Pat Possible. The author has a website [ https://00ff00.de/],and email [hallo@00ff00.de] to post problems and ideas. This page,  https://00ff00.de/?p=228, has more info on the Gadget.  Achtung, Heir Tuicemen, it might have an opening to another free API or service.  Oh, on second thought, hold grilling the bratwurst, I just searched through the 5Mb zipfile used for installation for clues. The author has a folder on OpenWeather.

I meant to mention also that if you really do want to learn about the inner workings of the Raspberry Pi I suggest you set one up from scratch. It's such a simple process really. Once you have the latest version of Raspbian installed a few simple copy/paste commands will install HomeGenie and HA Bridge too (if you feel you need Alexa). In approximately 1hr 30mins you will have a fully functioning home automation system. This can all be achieved without disturbing your current setup. This way you will never find yourself crippled waiting for 3rd party support. Once you start creating your own Wizard scripts the skies the limit.
Title: Re: WeatherAlerter died after HG v1.3 stable.13 update
Post by: Tuicemen on June 20, 2020, 01:16:15 PM
All programs & widgets posted for HG by me are ment to be expanded on and as a learning tool, not just for  me. The code is visible to be tweaked any way an end user sees fit. 
A word on the openweather API, the free version is limited to the number of calls you can make with a key once you go over that limit data will not be refreshed. That was one reason I used the info from the HG open weather rather the make a direct pull. If you have other programs runing on other machine nusing the same key your limit will be reached quicker. Your not limited to the number of keys you can have for the api so use a different key for different programs using the API.
As for the Duck DNS program and wiget, the displayed IP in the widget is not part of a DNS status call. (No need for DuckDNS registration)So users could modify the code and widget to only do a IP request and display just that.