Tag Archives: ubuntu

make Ubuntu server powerloss proof

Hi

Sometimes, after powerloss Ubuntu hangs on boot or asking to fix filesystem errors.
To avoid this problems simply edit following files:

Edit the file:

 /etc/default/grub
GRUB_RECORDFAIL_TIMEOUT=2

After that you have to rebuild grub configuration:

update-grub

Edit the file:

/etc/default/rcS
FSCKFIX=yes

EDIT 16.05.2018:
Ubuntu 18.04 /etc/default/rcS missing due to systemd
You have to pass a kernel parameter
https://www.freedesktop.org/software/systemd/man/systemd-fsck@.service.html

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash fsck.repair=yes"

Dont forget to run update-grub

Have fun!

rsyslog own mysql table per host with loganalyzer innodb and innodb-compression

Hi

Here my rsyslog config file placed under /etc/rsyslog.d/48-mysql.conf

$ModLoad ommysql
$template dbFormatHost1,"insert into SystemEventsHost1 (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, ProcessID) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag:R,ERE,1,FIELD:(.+)(\[[0-9]{1,5}\]).*--end%', '%syslogtag:R,ERE,1,BLANK:\[([0-9]{1,5})\]--end%')",SQL
$template dbFormatHost2,"insert into SystemEventsHost2 (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, ProcessID) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag:R,ERE,1,FIELD:(.+)(\[[0-9]{1,5}\]).*--end%', '%syslogtag:R,ERE,1,BLANK:\[([0-9]{1,5})\]--end%')",SQL
:source , isequal , "host1" :ommysql:localhost,Syslog,rsyslog,mypassword;dbFormatHost1
:source , isequal , "host2" :ommysql:localhost,Syslog,rsyslog,mypassword;dbFormatHost2

Here my database table structure

CREATE TABLE IF NOT EXISTS `SystemEventsHost1` (
 `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
 `CustomerID` bigint(20) DEFAULT NULL,
 `ReceivedAt` datetime DEFAULT NULL,
 `DeviceReportedTime` datetime DEFAULT NULL,
 `Facility` smallint(6) DEFAULT NULL,
 `Priority` smallint(6) DEFAULT NULL,
 `FromHost` varchar(60) DEFAULT NULL,
 `Message` text,
 `NTSeverity` int(11) DEFAULT NULL,
 `Importance` int(11) DEFAULT NULL,
 `EventSource` varchar(60) DEFAULT NULL,
 `EventUser` varchar(60) DEFAULT NULL,
 `EventCategory` int(11) DEFAULT NULL,
 `EventID` int(11) DEFAULT NULL,
 `EventBinaryData` text,
 `MaxAvailable` int(11) DEFAULT NULL,
 `CurrUsage` int(11) DEFAULT NULL,
 `MinUsage` int(11) DEFAULT NULL,
 `MaxUsage` int(11) DEFAULT NULL,
 `InfoUnitID` int(11) DEFAULT NULL,
 `SysLogTag` varchar(60) DEFAULT NULL,
 `EventLogType` varchar(60) DEFAULT NULL,
 `GenericFileName` varchar(60) DEFAULT NULL,
 `SystemID` int(11) DEFAULT NULL,
 `ProcessID` varchar(60) NOT NULL DEFAULT '',
 `checksum` int(11) unsigned NOT NULL DEFAULT '0',
 PRIMARY KEY (`ID`) KEY_BLOCK_SIZE=4,
 KEY `ReceivedAt` (`ReceivedAt`),
 KEY `Facility` (`Facility`),
 KEY `Priority` (`Priority`),
 KEY `FromHost` (`FromHost`),
 KEY `DeviceReportedTime` (`DeviceReportedTime`),
 KEY `SysLogTag` (`SysLogTag`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8 AUTO_INCREMENT=1 ;
CREATE TABLE IF NOT EXISTS `SystemEventsHost2` (
 `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
 `CustomerID` bigint(20) DEFAULT NULL,
 `ReceivedAt` datetime DEFAULT NULL,
 `DeviceReportedTime` datetime DEFAULT NULL,
 `Facility` smallint(6) DEFAULT NULL,
 `Priority` smallint(6) DEFAULT NULL,
 `FromHost` varchar(60) DEFAULT NULL,
 `Message` text,
 `NTSeverity` int(11) DEFAULT NULL,
 `Importance` int(11) DEFAULT NULL,
 `EventSource` varchar(60) DEFAULT NULL,
 `EventUser` varchar(60) DEFAULT NULL,
 `EventCategory` int(11) DEFAULT NULL,
 `EventID` int(11) DEFAULT NULL,
 `EventBinaryData` text,
 `MaxAvailable` int(11) DEFAULT NULL,
 `CurrUsage` int(11) DEFAULT NULL,
 `MinUsage` int(11) DEFAULT NULL,
 `MaxUsage` int(11) DEFAULT NULL,
 `InfoUnitID` int(11) DEFAULT NULL,
 `SysLogTag` varchar(60) DEFAULT NULL,
 `EventLogType` varchar(60) DEFAULT NULL,
 `GenericFileName` varchar(60) DEFAULT NULL,
 `SystemID` int(11) DEFAULT NULL,
 `ProcessID` varchar(60) NOT NULL DEFAULT '',
 `checksum` int(11) unsigned NOT NULL DEFAULT '0',
 PRIMARY KEY (`ID`) KEY_BLOCK_SIZE=4,
 KEY `ReceivedAt` (`ReceivedAt`),
 KEY `Facility` (`Facility`),
 KEY `Priority` (`Priority`),
 KEY `FromHost` (`FromHost`),
 KEY `DeviceReportedTime` (`DeviceReportedTime`),
 KEY `SysLogTag` (`SysLogTag`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8 AUTO_INCREMENT=1 ;

Here my loganalyzer source config snippet

$CFG['Sources']['Host1']['ID'] = 'Host1';
$CFG['Sources']['Host1']['Name'] = 'Host1';
$CFG['Sources']['Host1']['ViewID'] = 'SYSLOG';
$CFG['Sources']['Host1']['SourceType'] = SOURCE_DB;
$CFG['Sources']['Host1']['DBTableType'] = 'monitorware';
$CFG['Sources']['Host1']['DBType'] = DB_MYSQL;
$CFG['Sources']['Host1']['DBServer'] = 'localhost';
$CFG['Sources']['Host1']['DBName'] = 'Syslog';
$CFG['Sources']['Host1']['DBUser'] = 'rsyslog';
$CFG['Sources']['Host1']['DBPassword'] = 'mypassword';
$CFG['Sources']['Host1']['DBTableName'] = 'SystemEventsHost1';
$CFG['Sources']['Host1']['DBEnableRowCounting'] = false;
$CFG['Sources']['Host2']['ID'] = 'Host2';
$CFG['Sources']['Host2']['Name'] = 'Host2';
$CFG['Sources']['Host2']['ViewID'] = 'SYSLOG';
$CFG['Sources']['Host2']['SourceType'] = SOURCE_DB;
$CFG['Sources']['Host2']['DBTableType'] = 'monitorware';
$CFG['Sources']['Host2']['DBType'] = DB_MYSQL;
$CFG['Sources']['Host2']['DBServer'] = 'localhost';
$CFG['Sources']['Host2']['DBName'] = 'Syslog';
$CFG['Sources']['Host2']['DBUser'] = 'rsyslog';
$CFG['Sources']['Host2']['DBPassword'] = 'mypassword';
$CFG['Sources']['Host2']['DBTableName'] = 'SystemEventsHost2';
$CFG['Sources']['Host2']['DBEnableRowCounting'] = false;

If you’re paranoid you can modify your ubuntu 14.04 apparmor profile also, just add these lines to /etc/apparmor.d/usr.sbin.rsyslogd
and enforce rsyslog’s apparmor profile

# Add these for mysql support
/etc/mysql/my.cnf r,
/etc/mysql/conf.d/ r,
/etc/mysql/conf.d/** r,
/{,var/}run/mysqld/mysqld.sock rw,
/usr/share/mysql/charsets/Index.xml r,

Have fun!

boinc ubuntu 12.04 headless with apparmor

Hi

Under Ubuntu 12.04 amd64 the package dependencies are (seems to me) broken, it want’s  to install about 300MB packages. After fiddling arround with apt-get I found the proper option

apt-get --no-install-recommends install boinc-client

Now I want to activate my worldcommunitygrid.org account. Were doing this:

boinccmd --lookup_account http://www.worldcommunitygrid.org username yourpassword

Weg get an hash returned, and with the hash we do:

boinccmd --host localhost --project_attach https://www.worldcommunitygrid.org hash

That’s it. Now were computing for worldcommunitygrid.org
Since I’m a little paranoid, I don’t trust anything which downloads binaries and executes them on my computer automatically. Therefore is a solution to isolate the processes at kernel space. Apparmor. Here my profile:

# vim:syntax=apparmor
# Last Modified: Mon Jul 30 23:00:13 2007
#include <tunables/global>

/usr/bin/boinc {
  #include <abstractions/base>
  #include <abstractions/bash>
  #include <abstractions/nameservice>
  #include <abstractions/wutmp>

  /SYSV* mrw,
  /bin/dash ixr,
  /bin/uname ixr,
  /dev/ r,
  /dev/pts/ r,
  /etc/boinc-client/* r,
  /etc/gai.conf r,
  /etc/magic r,
  /lib/ r,
  /proc/ r,
  /proc/** r,
  /usr/lib/ r,
  /usr/bin/boinc mr,
  /usr/bin/file ixr,
  /usr/sbin/sendmail ixr,
  /usr/share/file/magic.mgc r,
  /var/lib/boinc-client/ r,
  /var/lib/boinc-client/** krwixr,
}

I’ve simply adopted the profile from:
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/129574
Save the profile under /etc/apparmor.d/usr.bin.boinc and run:

aa-enforce /etc/apparmor.d/usr.bin.boinc

If youre missing the command aa-enforce try to install the package apparmor-utils

Have fun!

ubuntu cgroup howto

Hi

Need to limit php-cgi procs?
Here some nice howto:

aptitude install cgroup-bin
service cgconfig stop

here the content of my /etc/cgconfig.conf

#mounts the cgroupfilesystems
mount {
   cpu = /sys/fs/cgroup/cpu;
   cpuset = /sys/fs/cgroup/cpuset;
   memory = /sys/fs/cgroup/memory;
}

#defines a group trasuser which is pinned to cpu 3 of the quadcore system
group trashuser {
   cpuset {
      cpuset.mems = 0;
      cpuset.cpus = 3;
   }
}

#defines the group defaultphp where all php-cgi processes get 75% of the quadcore cpu and 3GB Ram
group defaultphp {
   cpu {
      cpu.shares = 750;
   }
   memory {
      memory.limit_in_bytes = 3G;
   }
}

here the content of my cgrules.conf

#the unix user webuser01 gets pinned in group trashuser
webuser01        cpuset        trashuser/
#pinnes all php-cgi procs to group defaultphp
*:php-cgi    cpu,memory    defaultphp/
#pinnes all other procs to the default profile
*        *        default/
service cgconfig start

Here a nice link with resources

Hint:
If “service cgconfig restart” not works try to unmount all mounted cgroupfilesystems. You will find them under “cat /proc/mounts”
After that it should start.

Have Fun!

rsyslog as remote log server

Hi

Need to seperate logfiles on your syslog server on per remote ip basis and syslog facility?
Here my /etc/rsyslog.d/40-remote.conf

*.* /var/log/remote/all.log
$template DynFile,"/var/log/remote/%fromhost-ip%/%syslogfacility-text%.log"
:source , !isequal , "yourhostname" ?DynFile
:source , !isequal , "yourhostname" ~

If you want to seperate logfiles per hostname basis use this

*.* /var/log/remote/all.log
$template DynFile,"/var/log/remote/%HOSTNAME%/%syslogfacility-text%.log"
:source , !isequal , "youhostname" ?DynFile
:source , !isequal , "yourhostname" ~

Just replace yourhostname with the output of the command “root@yourhostname:~# hostname”

If you use Ubuntu 10.04 there is a bug in rsyslog with the option “$PrivDropToGroup syslog” in /etc/rsyslog.conf
Just replace the line with “$PrivDropToGroup adm”

And finally here my logroation config placed under /etc/logrotate.d/remote

/var/log/remote/*/*.log
/var/log/remote/all.log
{
        rotate 52
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                reload rsyslog >/dev/null 2>&1 || true
        endscript
}

Have fun!