##################
# timestamp: Simplified function returns a timestamp
# calling profile: mytimestamp=×tamp();
# returns: timestamp
##################
sub timestamp {
my ($flag,$message) = @_;
my $timestamp;
my $timedate;
my $date;
my $time;
my $sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst;
my $thisday, $thismon;
my $yy,$yyyy;
($sec,$min,$hour,$mday,$mon,$yy,$wday,$yday,$isdst) = localtime(time);
$thisday= (Sun,Mon,Tue,Wed,Thu,Fri,Sat)[$wday];
$thismon= (Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec)[$mon];
#If the year month, day, hour, minute, or second, are less than 10 prepend with a 0
$yyyy = $yy + 1900; #add 1900 to get 4 digit year
$yy -= 100; #subtract 100 to get 2 digit year
if ($yy < 10 ) {
$yy = "0$yy";
}
$mon++; #add 1 to month as month array starts at 0
if ($mon < 10 ) {
$mon = "0$mon";
}
# Day
if ($mday < 10 ) {
$mday = "0$mday";
}
# Hour
if ($hour < 10 ) {
$hour = "0$hour";
}
# Min
if ($min < 10 ) {
$min = "0$min";
}
# Second
if ($sec < 10 ) {
$sec = "0$sec";
}
# TO GET FORMAT USE THIS
#=================================================
# hh:mm:ss $timestamp="$hour:$min:$sec";
# hhmmss $timestamp="$hour$min$sec";
# dd/mm/yy $timestamp="$mday/$mon/$yy";
# mm/dd/yy") $timestamp="$mon/$mday/$yy";
# mm/dd/yyyy $timestamp="$mon/$mday/$yyyy";
# yyyymmdd $timestamp="$yyyy$mon$mday";
# yyyymmddhhmm $timestamp="$yyyy$mon$mday$hour$min";
# yyyymmddhhmmss $timestamp="$yyyy$mon$mday$hour$min$sec";
# yyyy-mm-dd $timestamp="$yyyy-$mon-$mday";
# mmddyyyy $timestamp="$mon$mday$yyyy";
# dd-mm-yy $timestamp="$mday-$mon-$yy";
# dd-mm-yyyy $timestamp="$mday-$mon-$yyyy";
# mm-dd-yy $timestamp="$mon-$mday-$yy";
# mm-dd-yyyy $timestamp="$mon-$mday-$yyyy";
# dd-mmm-yyyy $timestamp="$mday-$thismon-$yyyy";
# mmm dd, yyyy $timestamp="$thismon $mday, $yyyy";
#yyyymmddhhmm
$timestamp="$yyyy$mon$mday$hour$min";
return ($timestamp);
}
Monday, July 22, 2013
Monday, July 1, 2013
Apache Error [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
Problem: One day I woke up and my Apache server which had been running for several years was down and wouldn't restart. I checked the Apache error log and found the following error message:
[notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
This particular error message caused Uncle Google to speak in riddles, many of the forum posts that were found led to non answers. After fumbling around a bit, I noticed that another Apache log " nss_error_log" was being updated and displayed the following error messages:
[error] Certificate not verified: 'Server-Cert'
[error] SSL Library Error: -8181 Certificate has expired
[error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
This particular error message caused Uncle Google to speak in riddles, many of the forum posts that were found led to non answers. After fumbling around a bit, I noticed that another Apache log " nss_error_log" was being updated and displayed the following error messages:
[error] Certificate not verified: 'Server-Cert'
[error] SSL Library Error: -8181 Certificate has expired
[error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
Now we've found the root cause of our issue it is in a module called nss. I never noticed it before so another visit to Uncle Google actually turned up some interesting reading and several solutions. In short mod_nss is an alternative to mod_ssl it does a few things that mod_ssl doesn't do, and can run along side mod_ssl.
Since I'm not now an expert on mod_nss, I'll leave the gory details of how to set it up and what it does to these articles [What is mod_nss , On Setting up Mod_NSS ,More detail ] but suffice it to say chances are if you didn't know you were using it you may not need it. If you are inheriting a system you probably want to dig deeper into why it is there. Below I'm listing four possible solutions.
- Simply do as the error message above says add NSSEnforceValidCerts off to your nss.conf file which is usually located in /etc/httpd/conf.d. The downside to this is that your nss_error_log will continue to get messages like "SSL Library Error: -8181 Certificate has expired"
- Remove nss.conf from /etc/httpd/conf.d this will of course cause mod_nss to not be loaded.
- Uninstall/re-install your nss rpm modules on re-installation a new certificate will be generated and your problem will go away for a few more years
- Properly setup your nss db or trust store using the certutil command. (you can Google for more details)
Wednesday, May 29, 2013
Raspberry Pi Digital Picture Frame (Part 2)
9/15/2013 I have posted new code to handle an issue with Iphones. When sending a picture from an IPhone the picture is always named Photo.jpg, this causes pictures to be overwritten. The solution was two fold, 1) add a time stamp to each file, 2) Check to see the file already exists then add an additional, index if two timestamps are the same.
Notes:
Editing files: You can use either nano or vi if you know it. I used vi, but will be referencing nano in this post.
Sudo vs Root user: If you are comfortable working as the root (or super user) you can avoid typing sudo before every command by executing sudo bash
Gmail Setup:
If Mom doesn't have a Gmail account create one for her. Then follow these instructions for enabling IMAP.
Pi Setup:
After boot up you will be presented with a a menu to finish setup. The first thing you should do is expand the root partition to fill the SD card, set your keyboard layout, and your timezone. Be patient as this will take a couple of minutes. You also have the option to have the GUI desktop enabled at startup, you can set this to NO as we won't need it. However I did use the GUI to create the initial WI-FI connection, and installed most of the required software from a terminal window. For the rest of this post I will assume you are at a command line.
Log In:
The default username is pi and the default password is raspberry.
Network Setup:
Before you can install any software you need to be connected to the internet. If you are directly connected to your router with an ethernet cable execute the following command to see if you've been assigned an IP address.
sudo ifconfig
There are two groups of output one for eth0 the other for wlan0. For a cable connected Pi you should see an ipaddress beginning with 10 or 198 in the eth0 block. If you purchased a Wi-Fi dongle and don't see an ipaddress, follow these instructions.
1. edit the file /etc/network/interfaces
sudo nano /etc/network/interfaces
2. Delete everything in the file, then copy and paste the below text in instead. Then modify the last two lines to reflect your actual network name "ssid" and password (keeping the double quotes on both):
3. Save the file using CTRL-X to exit, Y to save the file and hit the Carriage Return to verify the filename.
4. Restart wlan0 by executing; sudo ifup wlan0
5. Verify you're router has given you an ipaddress by executing; sudo ifconfig
6. Verify internet connection by executing; sudo ping www.google.com
auto lo
iface lo inet loopback
iface eth0 inet dhcp
allow-hotplug wlan0
auto wlan0
iface wlan0 inet dhcp
wpa-ssid "ssid"
wpa-psk "password"
Notes: Some additional commands to help you debug.
Shutdown wireless connection: ifdown wlan0
List available networks: iwlist wlan0 scan
Install some software:
Hopefully everything worked and you are now connected to the web.
Install Perl Modules:
The Perl script that logs into your Mom's email account needs to call a few functions that are not already installed on your Raspberry Pi. Thus we need to install what are known as Perl modules. The tool we use is called cpan, the good folks who built the wheezy installation have included the cpan command in the installation. Once started the cpan tool will search the known online Perl module libraries for the modules and any dependencies we need and install them for us. To install the needed perl modules do the following. A couple of notes before you begin..
sudo cpan
cpan[1]> install MIME::Parser
....
cpan[2]> install Mail::IMAPClient
....
cpan[3]> install IO::Socket::SSL
....
cpan[4]> exit
Install the scripts
We are going to create two shell scripts and one Perl script. The shell scripts are short and can be typed in. The Perl script is rather large I list it at the bottom of this post or you can download it from the link below
Create the directory to hold your scripts and pictures.
sudo mkdir /usr/local/getpics
sudo chown pi:pi /usr/local/getpics
Create the scripts:
The first script is called downloadpic.sh, the second slideshow.sh the third picturedownloader.pl. Use nano to create your scripts. Remember to quit nano use CTRL-X, answer Y and then hit Carriage Return to confirm.
To download each script you can use the wget command as follows.
wget https://sites.google.com/site/relengetc/downloadpics.sh
wget https://sites.google.com/site/relengetc/slideshow.sh
wget https://sites.google.com/site/relengetc/picturedownloader.pl
Or enter them manually
cd /usr/local/getpics
nano downloadpics.sh
#!/bin/sh
cd /usr/local/getpics
/usr/local/getpics/picturedownloader.pl
sudo reboot
nano slideshow.sh
#!/bin/sh
cd /usr/local/getpics/pictures
/usr/bin/fbi -noverbose -t 10 -n 800x600 -a *
Add execution permissions to your scripts
chmod 755 downloadpics.sh slideshow.sh picturedownloader.pl
Finishing Up
The Raspberry Pi will prompt you for your username and password every time it boots up. For everything to work we need the "pi" user to be automatically logged in at boot time. We need to modify one of the system startup scripts.
sudo nano /etc/inittab
Find the line that looks like this and comment it out by placing a # in front of it.
1:2345:respawn:/sbin/getty --noclear 38400 tty1
It should look like this
#1:2345:respawn:/sbin/getty --noclear 38400 tty1
1:2345:respawn:/bin/login -f pi tty1 </dev/tty1 >/dev/tty1 2>&1
Save your changes (CTRL-X, Y, CR)
Next we need to enable the linux command scheduler called cron and start our slide show on startup
We need to modify another startup file
sudo nano /etc/rc.local
Scroll to the bottom then insert the following two lines.
/etc/init.d/cron/start
bash /usr/local/getpics/slideshow.sh
crontab -e
30 8,12,20 * * * /usr/local/getpics/downloadpics.sh 2&>1 /dev/null
Picture Downloader Script
The only change you will need to make to this script is to set the proper username and password on lines 6 and 7.
#!/usr/bin/perl
#Perl Modules needed
use MIME::Parser;
use Mail::IMAPClient;
#Connection information
my $username = 'username'; #your gmail username
my $password = 'password'; #your gmail password
my $mailhost = 'imap.gmail.com'; #Only change if not using gmail
my $debug = 0; #Set to 0 to turn off debugging
#Environment setup
# Directory where you want your pictures
my $outputdir = "./pictures";
# Directory where the message text and header information is placed
# Note: The contents of messagedir will be removed after picture download
# There may be a way to keep the message text ,header and extra copy of the imagei
# from being downloaded but I haven't figured it out.
my $messagedir = "./tmp";
#Setup some variables that we will need later
my (@body, $i, $subentity);
my ($x, $newx, @attachment, $attachment, @attname, $bh, $nooatt);
my $image="";
my $from="";
my $subject="";
my $timestamp="";
#These are the types of attachments allowed
#List of mime types found here: http://www.freeformatter.com/mime-types-list.html
#ex: If you want to add pdfs you would add application/pdf
my @attypes= qw(image/bmp
image/gif
image/jpeg
image/png
image/tiff
);
#Create ouput directories if they don't exist.
#Not the most efficient piece of code but effective and readable
if (! -d $outputdir ) {
if (!mkdir ("$outputdir", 0755)) {
die "Unable to create directory $outputdir\n";
}
}
if (! -d $messagedir ) {
if (!mkdir ("$messagedir", 0755)) {
die "Unable to create directory $messagedir\n";
}
}
#Connect to your mail host
my $imap = Mail::IMAPClient->new(
Debug => $debug,
User => $username,
Password => $password,
Server => $mailhost,
port => 993,
Uid => 1,
ssl => 1
) || die "Unable to connect to $mailhost \n";
my $newm=0;
$newm = $imap->select('INBOX');
#need to verify if this code actually works
if ($newm==0) {
$imap->logout();
print "No New Messages.\n";
exit;
}
my $parser = new MIME::Parser;
#my $error = ($@ || $parser->last_error);
$parser->output_dir($messagedir);
#Search SUBJECT lines for pattern "PIC"
#my @messages = $imap->search('SUBJECT',"PIC");
#Search for NOT SEEN emails
my @messages = $imap->search('NOT',"SEEN");
#print "Number of Messages: $#messages \n";
#Search thru each New message
foreach my $id (@messages) {
print "message id: $id\n" if ($debug);
my $entity = $parser->parse_data($imap->message_string($id));
#Lets see if we are able to parse anything
print "====================================================== \n" if ($debug);
chomp( $from = $entity->head->get('FROM') );
chomp( $subject = $entity->head->get('SUBJECT') );
chomp( $timestamp = $entity->head->get('Date') );
print " FROM: $from \n SUBJECT: $subject \n DATE: $timestamp \n" if ($debug);
print "====================================================== \n\n\n\n" if ($debug);
#Get email body borrowed most of the code
#from here: http://www.perlmonks.org/index.pl?node_id=195442
if ($entity->parts > 0){
for ($i=0; $i<$entity->parts; $i++){
$subentity = $entity->parts($i);
if (($subentity->mime_type =~ m/text\/html/i) || ($subentity-> mime_type =~ m/text\/plain/i)){
$body = join "", @{$subentity->body};
next;
}
#this elsif is needed for Outlook's nasty multipart/alternative messages
elsif ($subentity->mime_type =~ m/multipart\/alternative/i){
$body = join "", @{$subentity->body};
#split html and text parts
@body = split /------=_NextPart_\S*\n/, $body;
#assign the first part of the message,
#hopefully the text, part as the body
$body = $body[1];
#remove leading headers from body
$body =~ s/^Content-Type.*Content-Transfer-Encoding.*?\n+//is;
next;
}
#new attachment code start
#grab attachment name and contents
foreach $x (@attypes){
if ($subentity->mime_type =~ m/$x/i){
$bh = $subentity->bodyhandle;
$attachment = $bh->as_string;
push @attachment, $attachment;
push @attname, $subentity->head->mime_attr('content-disposition.filename');
}else{
#some clients send attachments as application/x-type.
#checks for that
$newx = $x;
$newx =~ s/application\/(.*)/application\/x-$1/i;
if ($subentity->mime_type =~ m/$newx/i){
$bh = $subentity->bodyhandle;
$attachment = $bh->as_string;
push @attachment, $attachment;
push @attname, $subentity->head->mime_attr('content-disposition.filename');
}
}
}
$nooatt = $#attachment + 1;
#new attachment code end
}
} else {
$body = join "", @{$entity->body};
}
#body may contain html tags. they will be stripped here
$body =~ s/(<br>)|(<p>)/\n/gi; #create new lines
$body =~ s/<.+\n*.*?>//g; #remove all <> html tages
$body =~ s/(\n|\r|(\n\r)|(\r\n)){3,}//g; #remove any extra new lines
$body =~ s/\ //g; #remove html   characters
#remove trailing whitespace from body
$body =~ s/\s*\n+$//s;
if ( $debug ) {
print "Messege was contructed as follows:
\$from: $from
\$to: $to
\$subject: $subject
\$body: $body
number of attachments: $nooatt
\$attachment(s): ".join ", ", @attname;
}
}
$imap->logout();
#write contents of each attachment to a file
for ($x = 0; $x < $nooatt; $x++){
$image = $attname[$x];
print "\n $x attachmentname:$image \n" if ($debug);
#strip one or more spaces in the image name and replace with _
$image =~ s/\s+/_/g;
$timestamp=×tamp();
$image="$timestamp"."_"."$image";
if ( -e "$outputdir/$image" ) {
$image="$x" . "_" . "$image";
print "\n hello: $image \n" if ($debug);
}
print "\n $x attachmentname:$image \n" if ($debug);
open FH, ">$outputdir/$image" || die "cannot open FH:$!\n";
print FH "$attachment[$x]";
close FH;
}
#cleanup
if (!opendir (TMPDIR, $messagedir)) {
die "unable to open $messagedir\n";
}
else {
@cleanup_list= grep !/^\.\.?$/, readdir(TMPDIR);
}
closedir(TMPDIR);
if ( defined @cleanup_list) {
foreach my $filename (@cleanup_list) {
if ( !unlink("$messagedir/$filename") ) {
warn("Unable to remove $messagedir/$filename\n");
}
}
}
##################
# timestamp: Simplified function returns a timestamp
# calling profile: mytimestamp=×tamp();
# returns: timestamp
##################
sub timestamp {
my ($flag,$message) = @_;
my $timestamp;
my $timedate;
my $date;
my $time;
my $sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst;
my $thisday, $thismon;
my $yy,$yyyy;
($sec,$min,$hour,$mday,$mon,$yy,$wday,$yday,$isdst) = localtime(time);
$thisday= (Sun,Mon,Tue,Wed,Thu,Fri,Sat)[$wday];
$thismon= (Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec)[$mon];
#If the year month, day, hour, minute, or second, are less than 10 prepend with a 0
$yyyy = $yy + 1900; #add 1900 to get 4 digit year
$yy -= 100; #subtract 100 to get 2 digit year
if ($yy < 10 ) {
$yy = "0$yy";
}
$mon++; #add 1 to month as month array starts at 0
if ($mon < 10 ) {
$mon = "0$mon";
}
# Day
if ($mday < 10 ) {
$mday = "0$mday";
}
# Hour
if ($hour < 10 ) {
$hour = "0$hour";
}
# Min
if ($min < 10 ) {
$min = "0$min";
}
# Second
if ($sec < 10 ) {
$sec = "0$sec";
}
# TO GET FORMAT USE THIS
#=================================================
# hh:mm:ss $timestamp="$hour:$min:$sec";
# hhmmss $timestamp="$hour$min$sec";
# dd/mm/yy $timestamp="$mday/$mon/$yy";
# mm/dd/yy") $timestamp="$mon/$mday/$yy";
# mm/dd/yyyy $timestamp="$mon/$mday/$yyyy";
# yyyymmdd $timestamp="$yyyy$mon$mday";
# yyyymmddhhmm $timestamp="$yyyy$mon$mday$hour$min";
# yyyymmddhhmmss $timestamp="$yyyy$mon$mday$hour$min$sec";
# yyyy-mm-dd $timestamp="$yyyy-$mon-$mday";
# mmddyyyy $timestamp="$mon$mday$yyyy";
# dd-mm-yy $timestamp="$mday-$mon-$yy";
# dd-mm-yyyy $timestamp="$mday-$mon-$yyyy";
# mm-dd-yy $timestamp="$mon-$mday-$yy";
# mm-dd-yyyy $timestamp="$mon-$mday-$yyyy";
# dd-mmm-yyyy $timestamp="$mday-$thismon-$yyyy";
# mmm dd, yyyy $timestamp="$thismon $mday, $yyyy";
#yyyymmddhhmm
$timestamp="$yyyy$mon$mday$hour$min$sec";
print "\n TIMESTAMP: $timestamp \n";
return ($timestamp);
}
Notes:
Editing files: You can use either nano or vi if you know it. I used vi, but will be referencing nano in this post.
Sudo vs Root user: If you are comfortable working as the root (or super user) you can avoid typing sudo before every command by executing sudo bash
Gmail Setup:
If Mom doesn't have a Gmail account create one for her. Then follow these instructions for enabling IMAP.
Enable IMAP in your Gmail settings
- Sign in to Gmail.
- Click the gear icon
in the upper right, then select Settings. - Click Forwarding and POP/IMAP.
- Select Enable IMAP.
- Click Save Change
Pi Setup:
After boot up you will be presented with a a menu to finish setup. The first thing you should do is expand the root partition to fill the SD card, set your keyboard layout, and your timezone. Be patient as this will take a couple of minutes. You also have the option to have the GUI desktop enabled at startup, you can set this to NO as we won't need it. However I did use the GUI to create the initial WI-FI connection, and installed most of the required software from a terminal window. For the rest of this post I will assume you are at a command line.
Log In:
The default username is pi and the default password is raspberry.
Network Setup:
Before you can install any software you need to be connected to the internet. If you are directly connected to your router with an ethernet cable execute the following command to see if you've been assigned an IP address.
sudo ifconfig
There are two groups of output one for eth0 the other for wlan0. For a cable connected Pi you should see an ipaddress beginning with 10 or 198 in the eth0 block. If you purchased a Wi-Fi dongle and don't see an ipaddress, follow these instructions.
1. edit the file /etc/network/interfaces
sudo nano /etc/network/interfaces
2. Delete everything in the file, then copy and paste the below text in instead. Then modify the last two lines to reflect your actual network name "ssid" and password (keeping the double quotes on both):
3. Save the file using CTRL-X to exit, Y to save the file and hit the Carriage Return to verify the filename.
4. Restart wlan0 by executing; sudo ifup wlan0
5. Verify you're router has given you an ipaddress by executing; sudo ifconfig
6. Verify internet connection by executing; sudo ping www.google.com
auto lo
iface lo inet loopback
iface eth0 inet dhcp
allow-hotplug wlan0
auto wlan0
iface wlan0 inet dhcp
wpa-ssid "ssid"
wpa-psk "password"
Notes: Some additional commands to help you debug.
Shutdown wireless connection: ifdown wlan0
List available networks: iwlist wlan0 scan
Install some software:
Hopefully everything worked and you are now connected to the web.
- Install any Raspberry Pi upgrades: sudo apt-get update
- Install Linux Frame Buffer image viewer: sudo apt-get install fbi
- Install perl ssleay libraries: sudo apt-get install libnet-ssleay-perl
Install Perl Modules:
The Perl script that logs into your Mom's email account needs to call a few functions that are not already installed on your Raspberry Pi. Thus we need to install what are known as Perl modules. The tool we use is called cpan, the good folks who built the wheezy installation have included the cpan command in the installation. Once started the cpan tool will search the known online Perl module libraries for the modules and any dependencies we need and install them for us. To install the needed perl modules do the following. A couple of notes before you begin..
- CPAN generates lots of output, you can ignore most of it.
- Linux and Perl are case sensitive it is important to enter your install commands exactly as listed below. If you mis-spell the module name cpan will not be able to find it.
- Answer "Yes" or "Y" when prompted to install any dependencies.
- Answer "NO" or "N" when asked if you want to run additional tests.
sudo cpan
cpan[1]> install MIME::Parser
....
cpan[2]> install Mail::IMAPClient
....
cpan[3]> install IO::Socket::SSL
....
cpan[4]> exit
Install the scripts
We are going to create two shell scripts and one Perl script. The shell scripts are short and can be typed in. The Perl script is rather large I list it at the bottom of this post or you can download it from the link below
Create the directory to hold your scripts and pictures.
sudo mkdir /usr/local/getpics
sudo chown pi:pi /usr/local/getpics
Create the scripts:
The first script is called downloadpic.sh, the second slideshow.sh the third picturedownloader.pl. Use nano to create your scripts. Remember to quit nano use CTRL-X, answer Y and then hit Carriage Return to confirm.
To download each script you can use the wget command as follows.
wget https://sites.google.com/site/relengetc/downloadpics.sh
wget https://sites.google.com/site/relengetc/slideshow.sh
wget https://sites.google.com/site/relengetc/picturedownloader.pl
Or enter them manually
cd /usr/local/getpics
nano downloadpics.sh
#!/bin/sh
cd /usr/local/getpics
/usr/local/getpics/picturedownloader.pl
sudo reboot
nano slideshow.sh
#!/bin/sh
cd /usr/local/getpics/pictures
/usr/bin/fbi -noverbose -t 10 -n 800x600 -a *
Add execution permissions to your scripts
chmod 755 downloadpics.sh slideshow.sh picturedownloader.pl
Finishing Up
The Raspberry Pi will prompt you for your username and password every time it boots up. For everything to work we need the "pi" user to be automatically logged in at boot time. We need to modify one of the system startup scripts.
sudo nano /etc/inittab
Find the line that looks like this and comment it out by placing a # in front of it.
1:2345:respawn:/sbin/getty --noclear 38400 tty1
It should look like this
#1:2345:respawn:/sbin/getty --noclear 38400 tty1
Now insert the line below
1:2345:respawn:/bin/login -f pi tty1 </dev/tty1 >/dev/tty1 2>&1
Save your changes (CTRL-X, Y, CR)
Next we need to enable the linux command scheduler called cron and start our slide show on startup
We need to modify another startup file
sudo nano /etc/rc.local
Scroll to the bottom then insert the following two lines.
/etc/init.d/cron/start
bash /usr/local/getpics/slideshow.sh
Finally we need to configure automatic downloads and a reboot by setting the crontab for the pi user. To do this we run the crontab -e command, which will use nano to open a file. Scroll to the bottom and add the line below. This will schedule your system to check for and download pictures if any at 8:30 am, 12:30pm and 8:30pm. Save as before (Ctrl-X, Y, CR)
30 8,12,20 * * * /usr/local/getpics/downloadpics.sh 2&>1 /dev/null
Picture Downloader Script
The only change you will need to make to this script is to set the proper username and password on lines 6 and 7.
#!/usr/bin/perl
#Perl Modules needed
use MIME::Parser;
use Mail::IMAPClient;
#Connection information
my $username = 'username'; #your gmail username
my $password = 'password'; #your gmail password
my $mailhost = 'imap.gmail.com'; #Only change if not using gmail
my $debug = 0; #Set to 0 to turn off debugging
#Environment setup
# Directory where you want your pictures
my $outputdir = "./pictures";
# Directory where the message text and header information is placed
# Note: The contents of messagedir will be removed after picture download
# There may be a way to keep the message text ,header and extra copy of the imagei
# from being downloaded but I haven't figured it out.
my $messagedir = "./tmp";
#Setup some variables that we will need later
my (@body, $i, $subentity);
my ($x, $newx, @attachment, $attachment, @attname, $bh, $nooatt);
my $image="";
my $from="";
my $subject="";
my $timestamp="";
#These are the types of attachments allowed
#List of mime types found here: http://www.freeformatter.com/mime-types-list.html
#ex: If you want to add pdfs you would add application/pdf
my @attypes= qw(image/bmp
image/gif
image/jpeg
image/png
image/tiff
);
#Create ouput directories if they don't exist.
#Not the most efficient piece of code but effective and readable
if (! -d $outputdir ) {
if (!mkdir ("$outputdir", 0755)) {
die "Unable to create directory $outputdir\n";
}
}
if (! -d $messagedir ) {
if (!mkdir ("$messagedir", 0755)) {
die "Unable to create directory $messagedir\n";
}
}
#Connect to your mail host
my $imap = Mail::IMAPClient->new(
Debug => $debug,
User => $username,
Password => $password,
Server => $mailhost,
port => 993,
Uid => 1,
ssl => 1
) || die "Unable to connect to $mailhost \n";
my $newm=0;
$newm = $imap->select('INBOX');
#need to verify if this code actually works
if ($newm==0) {
$imap->logout();
print "No New Messages.\n";
exit;
}
my $parser = new MIME::Parser;
#my $error = ($@ || $parser->last_error);
$parser->output_dir($messagedir);
#Search SUBJECT lines for pattern "PIC"
#my @messages = $imap->search('SUBJECT',"PIC");
#Search for NOT SEEN emails
my @messages = $imap->search('NOT',"SEEN");
#print "Number of Messages: $#messages \n";
#Search thru each New message
foreach my $id (@messages) {
print "message id: $id\n" if ($debug);
my $entity = $parser->parse_data($imap->message_string($id));
#Lets see if we are able to parse anything
print "====================================================== \n" if ($debug);
chomp( $from = $entity->head->get('FROM') );
chomp( $subject = $entity->head->get('SUBJECT') );
chomp( $timestamp = $entity->head->get('Date') );
print " FROM: $from \n SUBJECT: $subject \n DATE: $timestamp \n" if ($debug);
print "====================================================== \n\n\n\n" if ($debug);
#Get email body borrowed most of the code
#from here: http://www.perlmonks.org/index.pl?node_id=195442
if ($entity->parts > 0){
for ($i=0; $i<$entity->parts; $i++){
$subentity = $entity->parts($i);
if (($subentity->mime_type =~ m/text\/html/i) || ($subentity-> mime_type =~ m/text\/plain/i)){
$body = join "", @{$subentity->body};
next;
}
#this elsif is needed for Outlook's nasty multipart/alternative messages
elsif ($subentity->mime_type =~ m/multipart\/alternative/i){
$body = join "", @{$subentity->body};
#split html and text parts
@body = split /------=_NextPart_\S*\n/, $body;
#assign the first part of the message,
#hopefully the text, part as the body
$body = $body[1];
#remove leading headers from body
$body =~ s/^Content-Type.*Content-Transfer-Encoding.*?\n+//is;
next;
}
#new attachment code start
#grab attachment name and contents
foreach $x (@attypes){
if ($subentity->mime_type =~ m/$x/i){
$bh = $subentity->bodyhandle;
$attachment = $bh->as_string;
push @attachment, $attachment;
push @attname, $subentity->head->mime_attr('content-disposition.filename');
}else{
#some clients send attachments as application/x-type.
#checks for that
$newx = $x;
$newx =~ s/application\/(.*)/application\/x-$1/i;
if ($subentity->mime_type =~ m/$newx/i){
$bh = $subentity->bodyhandle;
$attachment = $bh->as_string;
push @attachment, $attachment;
push @attname, $subentity->head->mime_attr('content-disposition.filename');
}
}
}
$nooatt = $#attachment + 1;
#new attachment code end
}
} else {
$body = join "", @{$entity->body};
}
#body may contain html tags. they will be stripped here
$body =~ s/(<br>)|(<p>)/\n/gi; #create new lines
$body =~ s/<.+\n*.*?>//g; #remove all <> html tages
$body =~ s/(\n|\r|(\n\r)|(\r\n)){3,}//g; #remove any extra new lines
$body =~ s/\ //g; #remove html   characters
#remove trailing whitespace from body
$body =~ s/\s*\n+$//s;
if ( $debug ) {
print "Messege was contructed as follows:
\$from: $from
\$to: $to
\$subject: $subject
\$body: $body
number of attachments: $nooatt
\$attachment(s): ".join ", ", @attname;
}
}
$imap->logout();
#write contents of each attachment to a file
for ($x = 0; $x < $nooatt; $x++){
$image = $attname[$x];
print "\n $x attachmentname:$image \n" if ($debug);
#strip one or more spaces in the image name and replace with _
$image =~ s/\s+/_/g;
$timestamp=×tamp();
$image="$timestamp"."_"."$image";
if ( -e "$outputdir/$image" ) {
$image="$x" . "_" . "$image";
print "\n hello: $image \n" if ($debug);
}
print "\n $x attachmentname:$image \n" if ($debug);
open FH, ">$outputdir/$image" || die "cannot open FH:$!\n";
print FH "$attachment[$x]";
close FH;
}
#cleanup
if (!opendir (TMPDIR, $messagedir)) {
die "unable to open $messagedir\n";
}
else {
@cleanup_list= grep !/^\.\.?$/, readdir(TMPDIR);
}
closedir(TMPDIR);
if ( defined @cleanup_list) {
foreach my $filename (@cleanup_list) {
if ( !unlink("$messagedir/$filename") ) {
warn("Unable to remove $messagedir/$filename\n");
}
}
}
##################
# timestamp: Simplified function returns a timestamp
# calling profile: mytimestamp=×tamp();
# returns: timestamp
##################
sub timestamp {
my ($flag,$message) = @_;
my $timestamp;
my $timedate;
my $date;
my $time;
my $sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst;
my $thisday, $thismon;
my $yy,$yyyy;
($sec,$min,$hour,$mday,$mon,$yy,$wday,$yday,$isdst) = localtime(time);
$thisday= (Sun,Mon,Tue,Wed,Thu,Fri,Sat)[$wday];
$thismon= (Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec)[$mon];
#If the year month, day, hour, minute, or second, are less than 10 prepend with a 0
$yyyy = $yy + 1900; #add 1900 to get 4 digit year
$yy -= 100; #subtract 100 to get 2 digit year
if ($yy < 10 ) {
$yy = "0$yy";
}
$mon++; #add 1 to month as month array starts at 0
if ($mon < 10 ) {
$mon = "0$mon";
}
# Day
if ($mday < 10 ) {
$mday = "0$mday";
}
# Hour
if ($hour < 10 ) {
$hour = "0$hour";
}
# Min
if ($min < 10 ) {
$min = "0$min";
}
# Second
if ($sec < 10 ) {
$sec = "0$sec";
}
# TO GET FORMAT USE THIS
#=================================================
# hh:mm:ss $timestamp="$hour:$min:$sec";
# hhmmss $timestamp="$hour$min$sec";
# dd/mm/yy $timestamp="$mday/$mon/$yy";
# mm/dd/yy") $timestamp="$mon/$mday/$yy";
# mm/dd/yyyy $timestamp="$mon/$mday/$yyyy";
# yyyymmdd $timestamp="$yyyy$mon$mday";
# yyyymmddhhmm $timestamp="$yyyy$mon$mday$hour$min";
# yyyymmddhhmmss $timestamp="$yyyy$mon$mday$hour$min$sec";
# yyyy-mm-dd $timestamp="$yyyy-$mon-$mday";
# mmddyyyy $timestamp="$mon$mday$yyyy";
# dd-mm-yy $timestamp="$mday-$mon-$yy";
# dd-mm-yyyy $timestamp="$mday-$mon-$yyyy";
# mm-dd-yy $timestamp="$mon-$mday-$yy";
# mm-dd-yyyy $timestamp="$mon-$mday-$yyyy";
# dd-mmm-yyyy $timestamp="$mday-$thismon-$yyyy";
# mmm dd, yyyy $timestamp="$thismon $mday, $yyyy";
#yyyymmddhhmm
$timestamp="$yyyy$mon$mday$hour$min$sec";
print "\n TIMESTAMP: $timestamp \n";
return ($timestamp);
}
Tuesday, May 28, 2013
Raspberry Pi Digital Picture Frame (Part 1)
A few years ago I purchased a Kodak Pulse email addressable picture frame for my mother. Though a little expensive at $200 I thought this was a great product. It allowed my siblings and I to send pictures to Mom from our cellphones and other devices. It was fun for Mom as well as she could see pictures of the grandkids almost immediately after an event happened. Unfortunately the picture frame stopped connecting to my mothers WI-FI network and I was unable to fix it. I didn't want to spend another $200 thus I began looking at alternatives, old desktops/laptops/tablets and perhaps using flikr, or some other photo sharing service. I didn't want to be tied down to yet another online service. As I was researching alternatives the Raspberry PI B was announced, this was the perfect solution, small footprint, linux based and cheap, and I probably had stuff laying about the house.
Before I begin describing the build I'd like to give a shout out to Cameron Wiebe for his excellent post on building a Raspberry Pi picture frame some of the instructions below come directly from his article.
What I Built
A digital picture frame that automatically checks a gmail account for new messages then downloads any attached images and automatically displays them. I configured the gmail account for IMAP, and used a Perl script to query the account.
What I Used
Things I bought:
Raspberry PI B (512 MB): $39.99
NetGear USB wireless card. $25.00.
USB keyboard: $7
USB mouse: $3
Things I had laying around:
4GB micro SD card with a bunch of adapters. See pic below.
7" Polaroid monitor, left over from a two screen auto DVD player
Video cable
5V 700mA wall wart left over from an old Nokia cell phone
12 V wall wart to power the monitor borrowed from a USB switch.
4 port unpowered usb hub
Setting up the Raspberry Pi
I started with the instructions here , then went here to download Raspberry Pi Wheezy. I have a MacBook so I followed the excellent instructions here to copy the image to the SD card. Be patient the transfer takes a few minutes. Please read the instructions carefully. If you do it wrong you could trash your system.
Now plug everything in and power up. I was pleasantly surprised everything started up without any issues. Read Part 2 for complete details to get your Digital Picture Frame up and running.
Before I begin describing the build I'd like to give a shout out to Cameron Wiebe for his excellent post on building a Raspberry Pi picture frame some of the instructions below come directly from his article.
What I Built
A digital picture frame that automatically checks a gmail account for new messages then downloads any attached images and automatically displays them. I configured the gmail account for IMAP, and used a Perl script to query the account.
What I Used
Things I bought:
Raspberry PI B (512 MB): $39.99
NetGear USB wireless card. $25.00.
USB keyboard: $7
USB mouse: $3
Things I had laying around:
4GB micro SD card with a bunch of adapters. See pic below.
7" Polaroid monitor, left over from a two screen auto DVD player
Video cable
5V 700mA wall wart left over from an old Nokia cell phone
12 V wall wart to power the monitor borrowed from a USB switch.
4 port unpowered usb hub
Setting up the Raspberry Pi
I started with the instructions here , then went here to download Raspberry Pi Wheezy. I have a MacBook so I followed the excellent instructions here to copy the image to the SD card. Be patient the transfer takes a few minutes. Please read the instructions carefully. If you do it wrong you could trash your system.
Now plug everything in and power up. I was pleasantly surprised everything started up without any issues. Read Part 2 for complete details to get your Digital Picture Frame up and running.
Tuesday, May 7, 2013
Makespec.py: error: Requires at least one scriptname file
Recently I inherited a set of python scripts with corresponding windows executable files. To create the executable (.exe) file one can used the tool pyinstaller. The scripts were maintained by multiple people using a shared network drive. A development team on the other side of the world needed access to the network share. To give them access I would have had to work with the corporate network team, which isn't always easy. However, this team has access to our centralized SCM system. As a Release Engineer and Subversion admin, I felt that these scripts should be put into source control. I proceeded to place the scripts in Subversion without any issues. I then needed to create a Jenkins job to execute the pyinstaller scripts. I spent nearly a day hacking the build script, and using uncle Google searching for the error in the header. After hitting my head repeatedly against the wall I discovered the simplest of solutions, and probably the most obvious.
I simply changed this:
C:\apps\pyinstaller\Makespec.py --onefile --out=. --noconsole --name=cl_downloader downloader.py
Downloader.py
usage: python Makespec.py [opts] <scriptname> [<scriptname> ...]
Makespec.py: error: Requires at least one scriptname file
To this:
python C:\apps\pyinstaller\Makespec.py --onefile --out=. --noconsole --name=cl_downloader downloader.py
I'm hoping that this simple solution will keep someone else from getting a headache.
I simply changed this:
C:\apps\pyinstaller\Makespec.py --onefile --out=. --noconsole --name=cl_downloader downloader.py
Downloader.py
usage: python Makespec.py [opts] <scriptname> [<scriptname> ...]
Makespec.py: error: Requires at least one scriptname file
To this:
python C:\apps\pyinstaller\Makespec.py --onefile --out=. --noconsole --name=cl_downloader downloader.py
I'm hoping that this simple solution will keep someone else from getting a headache.
Monday, February 25, 2013
MySQL Replication
Configuring Master/Master replication
The following
procedure can be used to setup MySQL in a master/master replication
configuration. We'll call the two MySQL
servers Master1 and Master2. In a dual master setup each
server functions as both a master and a slave to the other server. The following assumptions
are being made:
·
You have two MySQL servers running the same MySQL
version.
·
Neither server is currently active (i.e. they are
not currently accepting inserts or deletions or modifications of any type.)
·
Master1 refers to the first server that has data
that you wish to replicate.
·
Master2 refers to the second server.
·
Master/Slave or Master/Master replication has not
already been implemented.
NOTE: Change IP
addreses and Server-ID’s in the following steps to match your environment.
1.
Configure
Master1 by adding the following to /etc/my.cnf
#Replication Setup
#set server id to the last octet of the
servers ip address
server-id = 150
replicate-same-server-id = 0
auto-increment-increment = 2
auto-increment-offset = 1
# change this to a path/name
appropriate to your system
log-bin = /usr/local/mysql/data/mysql-bin
log-bin-index =
/usr/local/mysql/data/mysqld-bin.index
relay-log = /usr/local/mysql/data/mysqld-relay-bin
relay-log-index =
/usr/local/mysql/data/mysqld-relay-bin.index
binlog-format = MIXED
binlog-cache-size = 1M
expire-logs-days = 7
sync-binlog = 1
skip-slave-start
log-slave-updates
2.
Configure
Master2 by adding the following to /etc/my.cnf.
#Replication Setup
#set server id to the last octet of the
servers ip address
server-id
= 148
replicate-same-server-id
= 0
auto-increment-increment
= 2
auto-increment-offset
= 2
# change this to a path/name
appropriate to your system
log-bin = /usr/local/mysql/data/mysql-bin
log-bin-index =
/usr/local/mysql/data/mysqld-bin.index
relay-log = /usr/local/mysql/data/mysqld-relay-bin
relay-log-index =
/usr/local/mysql/data/mysqld-relay-bin.index
binlog-format = MIXED
binlog-cache-size = 1M
expire-logs-days = 7
sync-binlog = 1
skip-slave-start
log-slave-updates
3.
Start MySQL on both servers.
Shell> cd
/usr/local/mysql
Shell> ./bin/mysqld_safe --user=mysql &
4.
Backup the Database on Master 1
Shell> mysqldump -u root -ppassword –-log-error=dump.log
--all-databases -–master_data > master1_snapshot_master_data.sql
5.
Capture the master_log_file name, and position from the newly
created backup
Shell>grep
MASTER_LOG_FILE master1_snapshot_master_data.sql | head -1
CHANGE MASTER TO
MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=390;
6.
Install the Database on Master2
Shell> mysql
–uroot –ppassword < master1_snapshot_master_data.sql
7.
Create
slave user on Master 1:
Adjust the IP address to reflect your systems.
mysql>
use mysql;
mysql> CREATE
USER 'slave_user'@'10.48.92.%' IDENTIFIED BY 'password';
mysql> GRANT
REPLICATION SLAVE ON *.* to 'slave_user'@'10.48.92.%' identified by 'password';
mysql> FLUSH
PRIVILEGES;
8.
Create
slave user on Master 2:
Adjust the IP address to reflect your systems.
mysql>
use mysql;
mysql> CREATE
USER 'slave_user'@'10.48.92.%' IDENTIFIED BY 'password';
mysql> GRANT
REPLICATION SLAVE ON *.* to 'slave_user'@'10.48.92.%' identified by 'password';
mysql> FLUSH
PRIVILEGES;
9.
Synchronize the Servers,
Master1:
mysql>
STOP SLAVE;
Master2: Set Master 2 to be a slave of Master 1
Set the value of “MASTER_LOG_FILE “ and “MASTER_LOG_POS” with the values discovered in step 5 above.
mysql>
STOP SLAVE;
mysql>
SHOW MASTER STATUS\G;
mysql>
CHANGE MASTER TO MASTER_HOST=’<MASTER1-IP>’, MASTER_USER=’slave_user’,
MASTER_PASSWORD=’password’, MASTER_LOG_FILE=’mysqld-bin.XXXXXX’, MASTER_LOG_POS=NNNN;
Master1: Set Master 1 to be a slave of Master 2
We don’t need the Position: value because NOPS only writes to MASTER 1.
mysql>
CHANGE MASTER TO MASTER_HOST =’<MASTER2-IP>’, MASTER_USER=’slave_user’,
MASTER_PASSWORD=’password’;
Master2:
mysql>
START SLAVE;
Master1:
mysql>
START SLAVE;
10. Verify Replication is running.
Execute the following on both
Master1 and Master2. For “SHOW SLAVE STATUS\G;” look for “Seconds_Behind_Master: 0”. For “SHOW PROCESSLIST;” look for “Master has sent all binlog to
slave; waiting for binlog to be updated”
mysql> SHOW SLAVE
STATUS\G;
…snip…
Seconds_Behind_Master: 0
…snip…
mysql> SHOW
PROCESSLIST;
…snip…
Master has sent all binlog to slave;
waiting for binlog to be updated
Slave has read all relay log; waiting
for the slave I/O thread to update it
…snip…
Sunday, January 6, 2013
SSH authentication order
I recently created a script the purpose of which is to copy statistics charts from one server to another. I had created ssh private/public key trust so that the script would be able to run automatically. However the connection to the remote server was in my opinion taking much too long. This was due to ssh running through all of the possible connection protocols in the default order. I figured there must be a way to change the order. After reading the man page for ssh_config I discovered that the default order is hostbased, publickey, keyboard-interactive, password. I also discovered that the order can be changed by changing the default order. SSH obtains configuration settings in the following order,
- command-line arguments
- user's configuration file ~/.ssh/config
- /etc/ssh/ssh_config
Host chartserver
HostName 12.34.56.78
PreferredAuthentications=publickey,hostbased,keyboard-interactive,password
Host setting can be an alias
Hostname can be DNS or IP address
PreferredAuthentications, change the default order to the order you prefer, in my case I wanted to use publickey as the first method as that would be the method I would be using. Now whenever I connect to the server the publickey authentication method will be the first that SSH attempts.
PreferredAuthentications, change the default order to the order you prefer, in my case I wanted to use publickey as the first method as that would be the method I would be using. Now whenever I connect to the server the publickey authentication method will be the first that SSH attempts.
Subscribe to:
Posts (Atom)