X10 Community Forum

📱🖥️PiX10Hub => 🏥Help and Trouble Shooting => Topic started by: mike on June 14, 2019, 08:09:05 AM

Title: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 14, 2019, 08:09:05 AM
Sometime in the last month my pi HG refuses to open in web browsers most of the time.  I can try switching between my 12 groups and occassionally one or two will open, but most of the time none do anymore.  Same on both browsers tried.

Any suggestions were to begin digging in?
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 14, 2019, 05:20:57 PM
Might be best if you post the problem with logs on the authors GitHub (don't hold your breath for a reply anytime soon  rofl ) or you could do the same on the homegenie.club forum where you'll probably have a better chance of resolving your issue. Incidentally have you done a Raspbian update recently. Raspbian Buster is due for release shortly and that should be fun  >!
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 17, 2019, 11:57:05 AM
I think it may be a cache related issue, but when my system does that, I log onto the HG RPi via SSH and reboot.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 17, 2019, 12:37:22 PM
Have rebooted many times.  Tried all addresses in browser (127.0.0.1, rasbpi, 292.168...:8080.  All 3 browsers...  hg is working fine, just cannot see groups. Can get to settings pages.   Just updated to latest hg.  No change.  Tried changing local host line in hosts file.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 17, 2019, 01:50:03 PM
Have rebooted many times.  Tried all addresses in browser (127.0.0.1, rasbpi, 292.168...:8080.  All 3 browsers...  hg is working fine, just cannot see groups. Can get to settings pages.   Just updated to latest hg.  No change.  Tried changing local host line in hosts file.

Mike you know logs are king when troubleshooting. Its a complete crap shoot without them

Launch HG manually from the terminal and watch what is happening as you work way through the browser. You'll probably resolve the matter yourself but if not post a log extract to whatever platform you choose for others to help.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 17, 2019, 03:37:38 PM
I see nothing glaring...
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 17, 2019, 05:09:40 PM
I see nothing glaring...

Are you calling an external service on your system. Do you have some form of bridge configured working with HG.

I did suggest posting an extract of your logs ie the ones that relate to the moments when HG starts acting up and not the whole log. Just watch the terminal and when you notice HG playing up copy/paste the relevant log transactions.

I've no idea what version of Mono you are using or what browser you use HG with. Are there particular groups that are causing these issues. When was the last time you updated Raspbian. Are your security certs valid.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 18, 2019, 05:20:56 AM
You've a lot of call traffic to the server. It looks like you have programs running there which you may not even be using. The Energy Monitor springs to mind. I imagine you still have most if not all the default programs that come with HG running. Disable the programs you don't use as a starting point.

Maybe if you post a screen cap of your Groups expanded so it can be seen what programs you have and which ones are active.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 18, 2019, 06:39:31 PM
I had deleted all the un ued HG programs, including energy monitor...   then redid my SD card from scratch some months ago and did not delete all those things.  I did turn off all the programs I am no using.  I still have HAbidge running on same rpi card:  until Gene comes back with opening up more bits send via rf, I am stuck using HAbidge to arm/disarm my security system.  Nothing else running. 

I need to update/upgrade pi - it has been months.  I have my cheat sheet with the commands somewhere around here to do it.

I have same no show performance on groups on ALL browsers.

You asked about sending logs:  I sent the only one I know of.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 19, 2019, 12:05:40 PM
IMO, I would not hold my breath for an update from Gene.  My experience has been that HG gets a bunch of updates clustered together when he has time/motivation to make some changes and then there is a long hiatus where nothing happens.  That used to be a week or a month when HG was being developed heavily.  In the last couple years, there were almost zero updates before Gene returned for the current round (last 6 months or so).

TBF, it's a free/open source product that Gene has never made any significant income on so it really shouldn't be his primary focus.  I think it's a great software for what it can do.  And in some ways (X10 RF) it is far superior to pretty much all other HA software.  It's a shame that his X10 interface can't be opened up to be used as an improvement for other tools but since it's based on .NET (Mono) it's not an option.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 19, 2019, 04:10:22 PM
It's important that when using programs outside of the standard HG package they are compiled properly and tested in the environment they are being used in before they go live. Just because something I design works for me it doesnt mean it will work for everyone else.

There are so many variations in hardware that can be used when running HG.You only have to look at the vast range of Raspberry Pi boards available and all the subtle differences between them and how they impact on the performance of HG. And that's only the start of the hardware range.

Then there's the different distributions of Raspbian, Wheezy (which should be retired by now but still in use by some) Jessie which has reached end of life and Stretch which will be shortly replaced by Buster.

Then there's the versions of Mono to consider that run on the above. We know well how this affects the likes of running an email notification when considering valid SSL certs. This problem has been highlighted both here and on other forums.

I could go on all day highlighting possible causes of instability with HG. That's why when reporting an issue with the software you should include the hardware its running on, the version of OS it's installed on and the version of Mono being used along with the relevant logs that reflect the issue as and when it's happening.

I highly recommend using the Raspberry Pi 3+ with the latest version of Raspbian fully updated and optionally a version of Mono that has proven to work satisfactorily with HG unless you only intend to run a basic X10 setup which in that case a Raspberry Pi Zero would do. Few choose to do this.

Again installing HG from official sources will help avoid installing any possible instabilities into your system. You can always import any external hgx files once they are proven to work on your setup along with any other configs that users contribute from time to time.

The above in my opinion is the best way to learn about HG and what it has to offer. It one of the easier software to install and configure albeit with the irregular and sporadic maintenance it receives from it maintainer as mentioned by @bkenobi . It's definitely ideal to get you started in the home automation world and an ideal primer for other open source home automation software.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 20, 2019, 09:50:28 AM


Then there's the different distributions of Raspbian, Wheezy (which should be retired by now but still in use by some) Jessie which has reached end of life and Stretch which will be shortly replaced by Buster.


What about updating stretch to Raspbian Pixel?   

https://www.cnet.com/how-to/update-your-raspberry-pi-to-raspbian-pixel-for-fancy-new-features/
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 20, 2019, 10:47:14 AM
I don't use a GUI with my RPi.  I'm strictly shell and happy with it for my HA needs.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 20, 2019, 10:55:52 AM
me too;  why I thought the pixel updates wouldn't do anything for me.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 20, 2019, 11:19:23 AM
BTW,

sudo apt-get update
sudo apt-get upgrade

gave a pretty scary message and made me enter q to continue:


wpasupplicant (2:2.6-19) unstable; urgency=medium

  With this release, wpasupplicant no longer respects the system
  default minimum TLS version, defaulting to TLSv1.0, not TLSv1.2. If
  you're sure you will never connect to EAP networks requiring anything less
  than 1.2, add this to your wpasupplicant configuration:

    tls_disable_tlsv1_0=1
    tls_disable_tlsv1_1=1

  wpasupplicant also defaults to a security level 1, instead of the system
  default 2. Should you need to change that, change this setting in your
  wpasupplicant configuration:

    openssl_ciphers=DEFAULT@SECLEVEL=2

  Unlike wpasupplicant, hostapd still respects system defaults.

 -- Andrej Shadura <andrewsh@debian.org>  Sat, 15 Dec 2018 14:22:18 +0100
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 20, 2019, 11:48:31 AM
So I found a sequence to make the groups show up (all but top one - dashboard):

1) config->group>pick one (NOT dashboard).  While it is showing,
2) click on home>pick a group (other than dashboard) and it pops up.  then all the others do also (except dashboard).

Appears what broke is in the dashboard section...  I will try to copy old dashboard section from a backup into present HG file groups.xml.   

no  luck yet
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 20, 2019, 12:50:04 PM
BTW,

sudo apt-get update
sudo apt-get upgrade

gave a pretty scary message and made me enter q to continue:


wpasupplicant (2:2.6-19) unstable; urgency=medium

  With this release, wpasupplicant no longer respects the system
  default minimum TLS version, defaulting to TLSv1.0, not TLSv1.2. If
  you're sure you will never connect to EAP networks requiring anything less
  than 1.2, add this to your wpasupplicant configuration:

    tls_disable_tlsv1_0=1
    tls_disable_tlsv1_1=1

  wpasupplicant also defaults to a security level 1, instead of the system
  default 2. Should you need to change that, change this setting in your
  wpasupplicant configuration:

    openssl_ciphers=DEFAULT@SECLEVEL=2

  Unlike wpasupplicant, hostapd still respects system defaults.

 -- Andrej Shadura <andrewsh@debian.org>  Sat, 15 Dec 2018 14:22:18 +0100

I'd say that message would have most users running for the q key as well  rofl. It's not that scary really.

You can now see the logic with staying updated regularly. Update snipers lying in wait for those who have their eye off the game.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 20, 2019, 12:59:47 PM
So I found a sequence to make the groups show up (all but top one - dashboard):

1) config->group>pick one (NOT dashboard).  While it is showing,
2) click on home>pick a group (other than dashboard) and it pops up.  then all the others do also (except dashboard).

Appears what broke is in the dashboard section...  I will try to copy old dashboard section from a backup into present HG file groups.xml.   

no  luck yet

Have you anything "non standard HG issue" added to your Dashboard. Maybe a screen cap of your Dashboard might help here. Also you might want to indicate the following

1. Raspberry Pi Model in use
2. Version of Raspbian
3. Version of Mono
4. Source from where HG was installed
5. What additional hardware attached to the Pi
6. Router model to which the Pi is attached to. Is it connected wired or wireless, is it behind a firewall or VPN

Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 21, 2019, 10:47:48 AM
How about this question?  Where is the active groups info stored?  I TOTALLY deleted the whole <Group> Dashboard section from groups.xml - yet it still shows up in HG list!
 
JUST like my problem with email parameter changes!  NOTHING I do seems to change the actions of HG!  There must be a secret location on the SD card where the REAL HG stuff runs from - it certainly is not from the HomeGenie directory!

I cannot DELETE the group in HG - it just locks up and won't show it under groups.  I can move other groups above dashboard - now on refresh the top group opens right away, all the other groups open fine, but as soon as I try to ope dashboard group - lock up....

Any clues?
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 21, 2019, 11:28:06 AM
I don't recall where it's stored at the moment.  I'm sure it's in one of the xml files though.  You can either manually search through them or use "grep" to find a key word (maybe dashboard).  There are around 6 files so you can do it either way pretty quickly.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 21, 2019, 11:30:36 AM
well...   finally was able to delete the misbehaving Dashboard group!  I had been leaving the group  /group in the groups.xml list, just deleted everythg in the group itself...  no difference...  finally just deleted the whole group from the groups.xml list in HG main directory - and it still showed up, complete with its unique background color.  BUT I as was able to finally config>group and open the group!  sofor kicks I tried tried first to delete the background color - it said deleted but it was still there.  then I picked delete whole group and it is gone.  now all my other groups work again.  sheesh...
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 21, 2019, 11:33:58 AM
I don't recall where it's stored at the moment.  I'm sure it's in one of the xml files though.  You can either manually search through them or use "grep" to find a key word (maybe dashboard).  There are around 6 files so you can do it either way pretty quickly.

Thanks.  That is how I have been finding the files.  I also copied the homegenie whole directory to my SSD HD so I could do fast searches for words INSIDE all files (like dashboard). 

Seems HG loads all the xml data into the running active program and uses it and does not see any changes made to the actual .xml files on the SD card; doing a restart of HG in the maintenance page does not reload the modified xml file either...
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 24, 2019, 06:27:14 PM
I don't recall where it's stored at the moment.  I'm sure it's in one of the xml files though.  You can either manually search through them or use "grep" to find a key word (maybe dashboard).  There are around 6 files so you can do it either way pretty quickly.

Thanks.  That is how I have been finding the files.  I also copied the homegenie whole directory to my SSD HD so I could do fast searches for words INSIDE all files (like dashboard). 

Seems HG loads all the xml data into the running active program and uses it and does not see any changes made to the actual .xml files on the SD card; doing a restart of HG in the maintenance page does not reload the modified xml file either...

Mike you do realise you can't just randomly drop files into folders in Linux. You have file permission and ownership to consider particularly when transferring between Windows and Linux as Windows does not recognise Linux ownership and permission. May account for the fact that when you delete a file is doesn't actually get deleted.

Again I know nothing of your setup it's just purely an observation. You may also want to bear that in mind when installing a downloaded file like an .hgx that's downloaded in Windows first. I'd advise that you stay in the Linux environment when working with HG on your Raspberry Pi. Just a basic piece of advice.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: Tuicemen on June 25, 2019, 09:09:19 AM
HG rebuilds the default groups on updates. So I'm wondering if this may be the cause of some issues? :o
Especially if one has removed some default things from their configuration.
Since HG tries to load the Dashboard first by default(unless you change the order of groups) if it is corrupt, HG fails to load correctly.

I don't run HG with the default groups and after a update I remove those recreated groups (Dashboard,ColorLights)
 >!
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 25, 2019, 09:23:31 AM
Thanks everyone for the help.  Yes, the issue is HG does NOT use any changes made until some secret formula is met.  Not knowing the secret formula, it is hard for me to get past issues.  I did nothing to damage my dashboard group, yet there it was broken.  Took a zillion changes before HG decided I met the requirements to reload the .xml file. 

I had the same issue trying to get my email working in the recent past.

I just went thru it again this week as I had to change out my router, thus requiring wpa_config reload for new ssid/pw - took a couple tries also.

I see no need for dashboard so did not put it back in;  like you Tuicemen, I justhave my groups.

Petera, I am well versed in CHOWN command. I do not "randomly drop files" or folders in Linux.  I use WinSCP and putty to go between win/pi.  My search skills are much better in windows and much faster on a SSD drive;  why I copy whole directories from my pi SD over to wind directories so I can do very fast searches of words within thousands of files in seconds.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 25, 2019, 05:40:44 PM
Of course it's your system and your choice how to configure it, but if I were you I would look into using SAMBA so you can keep the files on Linux and use all of your Windows tools natively on the files while still on the RPi.  It's easy to set up and works very well.  I included it in my setup script that I shared a while back.  I should probably copy the script over to my github so that others can use it rather than having to search for it here or on the HG club site.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 25, 2019, 06:20:26 PM
sounds intriguing...  I think I use Samba to force my wife's apple computers to be avail to my windows stuff;  works on all but one...  I would be interested in seeing what you do if you get a chance to point me to it sometime.  Thanks.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: bkenobi on June 26, 2019, 11:50:15 AM
This isn't my instructions, but should be about the same (looks similar to the guide I originally used).  It's the first link Google gave me for "samba raspberry pi".

https://www.raspberrypi.org/magpi/samba-file-server
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 26, 2019, 05:16:29 PM
Thanks everyone for the help.  Yes, the issue is HG does NOT use any changes made until some secret formula is met.  Not knowing the secret formula, it is hard for me to get past issues.  I did nothing to damage my dashboard group, yet there it was broken.  Took a zillion changes before HG decided I met the requirements to reload the .xml file. 

I had the same issue trying to get my email working in the recent past.

I just went thru it again this week as I had to change out my router, thus requiring wpa_config reload for new ssid/pw - took a couple tries also.

I see no need for dashboard so did not put it back in;  like you Tuicemen, I justhave my groups.

Petera, I am well versed in CHOWN command. I do not "randomly drop files" or folders in Linux.  I use WinSCP and putty to go between win/pi.  My search skills are much better in windows and much faster on a SSD drive;  why I copy whole directories from my pi SD over to wind directories so I can do very fast searches of words within thousands of files in seconds.

Great to hear that you are well versed with Linux ownerships and permissions. I really am puzzled why you are having so much difficulty with HG though. Clearly from what you now say you do realise that copying a downloaded file from Windows to Linux using SCP comes with its own warning

Simple curl or wget commands might make more sense but you seem to be on top of things so good luck with that Groups problem you are having.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: mike on June 26, 2019, 06:22:40 PM

1) Great to hear that you are well versed with Linux ownerships and permissions.
2) I really am puzzled why you are having so much difficulty with HG though.
3) Clearly from what you now say you do realise that copying a downloaded file from Windows to Linux using SCP comes with its own warning
4) simple curl or wget commands might make more sense but you seem to be on top of things so good luck with that Groups problem you are having.

Thank you for your useful? comments.  I will address each one individually simply because I am too old to play PC nonsense...

1) Sarcasm did not help anything.
2) Huh?  I had an issue many months ago setting up email and it was well documented that once initial parameters were entered no updates/changes to these were implemented.  Then last week my dashboard group stopped showing up, further causing all other groups to also not show up.  Again I could document that changes to groups.xml were NOT implemented until something else triggered a reread;  then my changes went into effect and the problem went away.  Not sure how these facts would cause you to be so puzzled.
3) Who ever said they copied files from windows to linux?  I go the other way for quicker easier diagnosis sometimes.
4) Groups problem I am having?  I explained quite some posts back that it was solved.
Title: Re: Where to investigate new case of HG not opening in browsers?
Post by: petera on June 26, 2019, 06:50:35 PM

1) Great to hear that you are well versed with Linux ownerships and permissions.
2) I really am puzzled why you are having so much difficulty with HG though.
3) Clearly from what you now say you do realise that copying a downloaded file from Windows to Linux using SCP comes with its own warning
4) simple curl or wget commands might make more sense but you seem to be on top of things so good luck with that Groups problem you are having.


Thank you for your useful? comments.  I will address each one individually simply because I am too old to play PC nonsense...

1) Sarcasm did not help anything.
2) Huh?  I had an issue many months ago setting up email and it was well documented that once initial parameters were entered no updates/changes to these were implemented.  Then last week my dashboard group stopped showing up, further causing all other groups to also not show up.  Again I could document that changes to groups.xml were NOT implemented until something else triggered a reread;  then my changes went into effect and the problem went away.  Not sure how these facts would cause you to be so puzzled.
3) Who ever said they copied files from windows to linux?  I go the other way for quicker easier diagnosis sometimes.
4) Groups problem I am having?  I explained quite some posts back that it was solved.

I do remember a while back spending many hours and posts helping you get set up on the Raspberry Pi with wpa_supplicant.conf. All documented here but what the hell. Like yourself I'm getting too old for all this patience stuff too.

I asked you in an earlier post to outline your setup but you chose not to. Thats why I wished you good luck but if you interpreted this in any other way that's your prerogative. Sarcasm is something I do not engage in. As you do know it's the lowest form of wit.

Not too many dishing out free advice these days but my last piece of free advice to you is to be a little bit more introspective before you jump to conclusions and a little thanks to people who give you free advice would go a long way. There's not too many people on this forum who actually work in the Linux world.