Almost every working day I've written a summation of the work as an intern at Kivo in Örnsköldsvik, Sweden.
The work I've done is a marketinganalysis about IoT, Systemdesign of a general IoT-surveillence system and troubleshooting the electronics of Kivos homealarm. Python, Linux, Raspberry pi, CAN, SPI, MySQL, PHP, Agile and workflows.
Today I've documented most of the classes so it's easy to understand and applicable to other programs. I also polished and made so that the alarm connects to a webcam and saves a picture on the r-pi. I didn't have time to add it to the email.
ssh with passwords and autossh is a bit tricky, there are a lot of small things to learn. I guess calling someone who knows would save time!
Tried to make the alarm start automatically when r-pi boots up, but can only do linux-commands and not start the pythonprogram... Hmm.
Tomorrow I'll do a transfer of the work to Anton so he knows the system. Maybe I'll have time to make the alarm startup automatically when booted, then it feels like a running prototype that kivo actually can use.
Today I worked on getting an SMS from a site when it notices that the raspberry pi hasn't uploaded data to a database at the expected time.
I also fixed the email, but not wil really logging into Bineros account (i think), but just usting a few lines from here. So extremely simple.
So now, if the r-pi now lose connection, in five minutes I will get an SMS and Email telling me it got offline. I'm turning it on now before I go home for today and I expect the SSH-tunnel to break in 12 hours, so in the middle of the night I will get an SMS. (I allways put my phone on silent during the night, so I can sleep).
Feels kinda good, tomorrow I'll try fixing the autossh so that the tunnel goes on and make the r-pi start everything up when it's started. After that I'll focus on documentation for the rest of the LIA. I haven't got any mail from Binero about fixing an autossh-tunnel, have they forgotten me?
There is still much I haven't done and can be improved. I have learned sooo much. Especially about SMS, Email, PHP, Linux, r-pi and electronic.
If the raspberry pi doesnt upload data within 1 minute, it reacts.
I have not yet fixed the mail-sending, sms-sending or uploading alarmtime. But it calculates a correct deltatime since the r-pi been up and reacts.
Tomorrow I'll focus on making the SMS part and uploading alarmtime done. The alarmtime is the time when this site sends the Offline warning. If that isn't saved somehow, the site will continually send sms. We just want one sms.
I couldn't fix the TLS error by myself and I thought I'd buy some time by calling Binero about the problem. We talked and then he started looking at my uploaded code, he said he was going to mail me about the problem so I don't have to wait in the phone. 2 hours later nothing, so I'll check tomorrow.
In the mean time I focused on writing the rapport for school and connected r-pi to the webcam to get an image, this image is going to be sent by email if there is an alarm. Neat?
Somehow when I tried to connect and try what I tried yesterday, it worked. Instead of getting:
_mysql_exceptions.OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0")
I got the name of the database version used (I sent the SQL "SELECT VERSION()") I checked the connections with netstat and ps aux and it seemed like yesterday. I tried to replicate the error but it allways worked now. So I went to upload some mysql. I got stuck like half an hour because I forgot commit()... But then I got it to work so now the database is updated every 10 min with sensordata. So now I can fully focus on the server-part. (I'm going to have problems in the future with SSH and if I have time I'll look into autossh to fix this) Then I started to look at sending email from the server again. No significant progress but I narrowed it down to that TLS seems not to be started by my PHP-script. So Tomorrow I'll be checkin' if that's the solution to the error:
I am also VERY happy of my troubleshooting documenting system I talked about in a previous post. This helps SO much to have all the relevant history. I also think it helps me focus about the right concepts and remember to forget about things that is not important. Get it?
Made MySQL program in python and it works with my windows laptop. I use putty to make the SSH-tunnel. But on my r-pi with linux i need to write stuff like:
And it makes me so confused. I managed to understand a lot but I think I have to learn a lot more before I get this to work... Going to continue with it tomorrow anyhow. Maybe I'll post on stack overflow for this one.
Got an answer from Binero about how to create an online checking alarm-system thing. A site that checks each halfhour if a system on another place is online, connected to the internet and alive and well. Apperently they offer a service to schedule running of URLs. This is perfect. This means I just need to make a site that checks if the remote alarm is okay and if not, it sends email and sms.
The Raspberry pi updates a MySQL database every halfhour or 5 min.
The online server checks if the r-pi has reported that it's okay.
This way we do not need to worry about opening ports on a router. This is important if we encouter a customer with several routers and we probably never need to put effort in configuring a router.
So I focused on making a site to send email. I choose to use PHP since I'd like to have a little more experience on that. But later I think I maybe should have used javascript..well well.
I learned that the server on binero can be run just like a raspberry pi using SSH to connect. I had to do this to install PEAR for PHP to send email. Haven't got it to work yet, big trouble connecting in the right way.
Learned some more PHP on codecademy really great site!
Tomorrow I'll be trying to fix this even more and then go on to make a POST to cellsynt to send SMS.
Presented my work to Andreas and Anton, and it was appreciated. They where also interested in how I work using a winscp - script to load up files fast and work with wing to program. Also the dataplicity connection making it available from anywhere was also interesting. I'm going to help them to set those things up so they know how.
I have also tried to set up the r-pi to run a command and the python program at startup but have not yet succeeded. Will continue with that tomorrow and then focus on making an online server checking data on a database. The pi will upload status every 5 minutes and the online server will check each half hour or so to see if there's been an upload the last half hour, if it hasn't, it will send sms or email telling that the r-pi might be down and not able to send alarms.
If I would make a server that connects to the r-pi, it would need to act as a webbservice and we would have to open ports on the router.
But doing it with a database means we don't have to open any ports on a router so if the system is to be made for a customer we don't have to manage a lot of routers, because there could be several routers on routers and legal things and other things that prevents from ports beeing opened.
Tomorrow, online server. It's nice that I've set up the other stuff for email and sms, so it should be pretty easy to have the server do that as well.
Today I've worked out the bug from last week. I just realized that it shouldn't be fixed until it is discovered when making a system for a customer. The problem is that it will send alarms to often. I don't know how to fix it without breaking the readability of the code, and I have not much time. I have prioritized other things more important.
I've started thinking about the online service that will check if the system is online and working.
I have also tidied up the code, made a map so it's easier to understand the code.
Tomorrow I will show what I've worked on and what needs a configuration and how the system can be adapted to other alarms. But I'm not satisyed because it's pretty messy still. Maybe I should have focused on making it easy to understand instead of making it modular. I will also continue working on the online service.
Today I've made classes for sending mail and sms. So now that works.
If a different mail-host than gmail should be used the class must change.
cellsynt provides a service where each sms costs 0,50 kr. That seems fitting since the alarms should not be sending a lot of sms:es, unless a place is frequently robbed?
Sms are sent using cellsynt through their API, very simple. Here's my code, just change the user, pass and originator to fit your needs. Se the document for what originator should be.
import requests
class SMSHANDLER():
"""Check https://www.cellsynt.com/pdf/Cellsynt_SMS_gateway_HTTP_interface_(Swedish).pdf
for more info on cellsynt web service"""
"""sends sms to destination. Destinations first 2 digits "00" followed
by landcode. For a swedish number "070 112233" write "004670 112233.
content should be UTF-8 (stick to english lanugage)"""
That was all that was needed to send an SMS! Amazingly simple. Also, you need to register an account and fill it up with sms. 100 sms is the smallest amount to fill, but I think you start out with 20 sms since I had 120 after i did a refill. Didn't check the quota before that.
In the last minute a bug arised today that means some things has to be rethought. The frames from the CAN-card seems to save frames for later or so, so it needs to be emtied or something, otherwise the alarm will sound like 10 times eash time there is an alarm. I'll check into that on monday. Then I'll configure it for Kivos purposes and make an raspbian img file along with documents probably.
It only has two methods that should have the same name for any alarm-type. The first method receives the CAN data and has to parse it to a nice format. The other method extracts the right data given a sensor-adress. This sensor adress can be anything uniqe.
3. The last thing is to edit alam_specific_methods.py. For this you need to know the basic system and python-language.
If a customer needs an alarm based on several sensor-values, alarm_specific_methods.py may need a class or global variables to keep track of data from other sensors.
If the hardware or communication protocols are different from what we use, CAN_communication.py and other files may need to change.
Tomorrow I will either focus on SMS, EMAIL, LOCAL SERVER to show data or trying to fix the broken part of the hardware. It depends on what the product owner wants.
Today I've been pythoning all day. Made everything I've already done but in a general way. So that it's easily applyable for other types of alarms.
General setup so far:
So the system should just be copied to any type of alarm that uses CAN.
The only files that will need be changing (hopefully) is the txt-files containing alarms and user information along with alarm_specific_methods.py. The second one (alarm_specific_methods.py) will parse the incoming data from CAN_communication and read data from alarm-txt files. If there is some threshhold that is reached or triggered, then different methods are run in the lowest three py-files. A customer could choose to get an email for example. And if the customer would change email, it's very easy to change the email in the txt-file.
Some things might need chaning in CAN_communication or other files if there is some update somewhere or if the system is using a different kind of CAN-protocol or so. I don't really know that stuff yet.
Tomorrow, more python and focus on alarm_specific_methods and what Andreas want to happen and thressholds and stuff.
The SPI data is fluctuating in a werid way. Today I spent the morning to get the SPI data to be shown live. I tried a bunch of tutorials to get tables and installed tabulate for python. In the end I had not time to get into ncurser to do this, so I just went around and opened windows and tried to 'see the matrix'.
I am drawing the conclusion that the system is supposed to show FF for a room if the windows are closed, and 00 if they are open. This works for some rooms but a specific can-card (210) is not showing acceptable data. I have no idea what 230 is suppose to show us and there are 4 values on 220 that's always 00. The four FF on 220 is working.
we decided to leave that card responsible for the 210 alone, at least for now. So I will develop the system and just ignore 210.
After lunch I focused on reading up on MySQL, making a table for the values and starting to plan the python-code for the r-pi.
Tomorrow I will probably just write a lot of very beautiful and newbie-friendly python-code. So it's easy for others to understand and build upon. After that, I will try to make it easy to install the whole system on an SD card, with everything there and document how to easily set up the system for any type of alarm. After that I will either look at making a website to constantly check if the r-pi is online, maybe an interface. And talk about sending SMS from server. Well let's not get ahead of ourselves.
I only have 2 weeks left here, I think I'll be able to program the r-pi and make a very simple server to check if the r-pi is online. Maybe I'll have time to send SMS, but email should be pretty easy.
pi@piv2:~ $ candump can0
This made the card constantly dump a lot of info. And one of the parts reacted when somebody moved in from of the motion detector.
After that, I managed to connect it with python using this tutorial. I chose to use SOCKETCAN and that worked well. I had to use Python 3 though, but that was a simple change.
I made a simple program that is run on the r-pi. When something moves, it simply says:
"MOTION DETECTED! ENGAGE LASERS! PEW PEW PEW!"
That was last week. Then I started to plan how the whole code should be structured.
Today, except for thinking about the structure, I've worked from home. I tried to work via dataplicity but coding python using that console is painfully slow. I hadn't set up SSH connection globally so I tried to do it from home using dataplicitys console. Didn't manage. I think the problem was mainly to open a port on the router. But I learnd a lot and maybe I'll fix it when I get back to work. But tomorrow I will probably focus on making a nice python program that shows like a live update of the SPI values. Not a spamming of the data, more like it rewrites the row so you can see when something changes.
This will help me find out what values are connected to what sensor. I think there is a problem with one/a few of them. What troubles me is that I really lack the skills to fix that, but it's interesting and I might learn a lot.
After that, maybe I can focus on making a nice replicable internet-surveilecne-part without the complete functioning of the alarm. So it can be used on all the other projects. Or upload some values to a database.
Btw, Dataplicitys idea is really cool. Maybe something that can be improved upon.
I put the canusb on hold just to tie up some loose ends. I tried this tutorial
Installed some can-utils and rebooted.
I followed the steps until 7. And at first it didn't work, I just got 'garbage' when I wrote candump can0
But After some time and looking at other stuff, maybe a reboot? I got an answer!
It continually spits out data now, the oscilloscope reacts and at least one piece of data makes sense. The one with red background turns to 00 if the PIR motion detection reacts.
pi@raspberrypi:~ $ candump can0
can0 220 [8] FF FF FF FF 00 00 00 00
can0 230 [8] 00 00 00 00 00 00 00 00
can0 210 [8] 00 00 00 00 00 00 00 00 continues to loop the 3 values forever... So that's pretty cool. Tomorrow I'm going to look at python to see if I can make some kind of program to react and maybe upload some data on an sql server or so. I've started to look at Python-can to do this.
The Raspberry pi updates a MySQL database every halfhour or 5 min.
