Koha Test Wiki MW Canasta on Koha Portainer

Test major Koha Wiki changes or bug fixes here without fear of breaking the production wiki.

For the current Koha Wiki, visit https://wiki.koha-community.org .

Plack

From Koha Test Wiki MW Canasta on Koha Portainer
Jump to navigation Jump to search

PSGI/Plack is an interface between Perl web applications and web servers, and Plack is a Perl module and toolkit that contains PSGI middleware, helpers and adapters to web servers. Inspired by Rack for Ruby and WSGI for Python, and it is the project behind the PSGI specification used by other frameworks such as Catalyst and Dancer.

Current state of Plack (December 2017)

Plack is recommended for use with current versions.


Plack on packages (3.22+)

If you want to enable Plack on the 3.22 packages you need to: 

  $ sudo koha-plack --enable <instance>
  $ sudo koha-plack --start  <instance>
  $ sudo service apache2 restart

Note: it currently requires Apache 2.4.10+, so it will work on Debian Jessie, but not on Ubuntu 14.04 or Debian 7. On this distributions the koha-plack --enable command will have no effect; unless you backport a newer Apache version.


State of plack (March 2015)

Breaking news: Rest of this page is very obolete, but right now (as of Koha Hackfest 2015 in Marseille) we are working on having plack as default option, so following are notes from this discussion....

See Bug 13791 - Plack out of the box

  • caching problems with systempreferences and framework
    • migrate to Koha::Cache library
    • Robin already tried to do this in http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11998, but it is somewhat big diff
    • as a stop-gap measure we can use C4::Context->disable_syspref_cache(); in koha.psgi but we have more than 200 cache requests for each search result page, so it's sub-optimal
    • another stop gap measure would be to write plack middleware which would call C4::Context->clear_syspref_cache(); on each request like this:
    enable sub {
        my ( $app, $env ) = @_;
        return sub {
            my $env = shift;
            C4::Context->clear_syspref_cache();
            $app->( $env );
        }
    };

dpavlin is currently using this approach in production without problems.

  • missing init script for plack startup for each instance (requeres apache 2.4, but we are depreciating squeeze anyway)
    • Use filesystem sockets (UDS) instead of ports for opac and intranet plack (DONE)
    • koha-common should start plack just like it does launch indexer and sip now (WIP)
    • psgi goes into /etc/koha/plack.psgi (DONE)
    • init script looks for optional per-instance psgi in /etc/koha/sites/instance/plack.psgi (WIP)
    • add function in koha-functions.sh (DONE)
    • koha-plack {--start|--stop|--restart} script (DONE)
    • koha-plack {--enable|--disable} instance (WIP, working POC, uses "Define plack + IfDefine...")
    • assume http opac and intranet, add commented out lines for https support (WIP)

Look here for the current code

    • dpavlin, thomas
  • during update we don't see what was applied in browser (See Bug 13793)
    • store result of updatedatate.pl into logfile /var/log/koha/instance/update.log
    • jonathan
  • Koha installer with empty database doesn't work with plack
    • work-ardound is to always run installer as cgi using ScriptAlias

Old information, somewhat outdated

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7172 tracks progress of plack changes in master - and most are already there

This page will try to document our efforts to run Koha using plack to gain persistence (so we don't have to load all Koha modules on each invocation as we do with cgi).

If you don't want to type, or if you are interested how we arrived at following snippets, code is available at https://github.com/dpavlin/Koha/tree/plack

I will rebase this repository and do nasty things to it, but if you just want to try out all plack changes together it might be a good place to start.

Installing plack

cpan App::cpanminus
cpanm Task::Plack
cpanm Plack::Middleware::Debug::Profiler::NYTProf

Or with packages 

sudo apt-get install libplack-perl libcgi-emulate-psgi-perl libfile-pushd-perl libtext-microtemplate-perl libclass-method-modifiers-perl \
libcss-minifier-xs-perl libjavascript-minifier-xs-perl 

sudo dh-make-perl --install --cpan CGI::Compile
sudo dh-make-perl --install --cpan Module::Versions
sudo dh-make-perl --install --cpan Plack::Middleware::Debug
sudo dh-make-perl --install --cpan Plack::Middleware::Static::Minifier
sudo dh-make-perl --build --cpan Plack::Middleware::Debug::DBIProfile

Running OPAC

This instructions assume that your Koha checkout is /srv/koha. If it's in some other location, you have to change /srv/koha to correct folder throughout all scripts below.

Create following file and call it opac.psgi 

#!/usr/bin/perl
use Plack::Builder;
use Plack::App::CGIBin;
use Plack::Middleware::Debug;
use Plack::App::Directory;
use lib("/srv/koha");
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;
use C4::Branch;
use C4::Category;

my $app=Plack::App::CGIBin->new(root => "/srv/koha/opac");

builder {
        enable 'Debug',  panels => [
                qw(Environment Response Timer Memory),
#               'Profiler::NYTProf', # FIXME this should work, but is missing .gif exclude
                [ 'Profiler::NYTProf', exclude => [qw(.*\.css .*\.png .*\.ico .*\.js .*\.gif)] ],
#               [ 'DBITrace', level => 1 ],
        ];


        enable "Plack::Middleware::Static",
                path => qr{^/opac-tmpl/}, root => '/srv/koha/koha-tmpl/';

        enable 'StackTrace';
        mount "/cgi-bin/koha" => $app;

};

For development, easiest way to test your new plack installation is using plackup: 

KOHA_CONF=/etc/koha/sites/ffzg/koha-conf.xml PERL5LIB=/srv/koha sudo -u ffzg-koha -E plackup --reload opac.psgi


Open http://localhost:5000/cgi-bin/koha/opac-main.pl and enjoy your new plack opac!

Running OPAC under plack is well tested by BibLibre and seems to work well with current master code.

Video of speed comparison between CGI and plack for OPAC: http://blog.rot13.org/koha/koha-opac-cgi-vs-plack.ogv

Running Intranet

Intranet support for plack is experimental. While we didn't find any problems (yet), it's not tested as well as opac.

Idea is to run intranet under separate plack server, since it will probably be on different port and/or protected via SSL.

Second day of plack testing showed that we are somewhat away from goal of running intranet under plack. Currently biggest problems are variabled declared with my which are inside sub (since plack wraps our cgi scripts within sub). This problem is similar to mod_perl problem described at http://modperlbook.org/html/6-2-Exposing-Apache-Registry-Secrets.html (just ignore mod_perl references :-)

There is a bug tracking all patches needed to run intranet under plack: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7172

Most of the patches are simple changes which change scoping of variables used within subs inside *.pl files. This is needed because pack wraps each CGI script into sub. While this approach may seem like a cludge, as we are creating global variables this doesn't affect CGI since all variables are released on end of request. In case of plack, this will increase in-memory size of plack a little bit, but it's simplest way (IMHO :-) to get code running on plack while preserving compatibility with CGI execution of Koha.

Create file intranet.psgi 

#!/usr/bin/perl
use Plack::Builder;
use Plack::App::CGIBin;
use Plack::Middleware::Debug;
use Plack::App::Directory;
#use Plack::Middleware::Debug::MemLeak;
use lib("/srv/koha");
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;
use C4::Branch;
use C4::Category;

my $app=Plack::App::CGIBin->new(root => "/srv/koha/");

builder {
        enable 'Debug',  panels => [
                qw(Environment Response Timer Memory),
#               [ 'Profiler::NYTProf', exclude => [qw(.*\.css .*\.png .*\.ico .*\.js .*\.gif)] ],
#               [ 'DBITrace', level => 1 ],
        ];

        enable "Plack::Middleware::Static",
                path => qr{^/intranet-tmpl/}, root => '/srv/koha/koha-tmpl/';

#       enable "Plack::Middleware::Static::Minifier",
#               path => qr{^/intranet-tmpl/},
#               root => './koha-tmpl/';

        enable 'StackTrace';
        mount "/cgi-bin/koha" => $app;

};

and run it using: 

KOHA_CONF=/etc/koha/sites/ffzg/koha-conf.xml PERL5LIB=/srv/koha sudo -E -u ffzg-koha plackup --reload --port 5001 intranet.psgi

Open http://localhost:5001/cgi-bin/koha/mainpage.pl and login into intranet.

Video of speed comparison between CGI and plack for intranet: http://blog.rot13.org/koha/koha-intranet-cgi-vs-plack.ogv


Deploying with starman

Starman is multi-process plack web server well suted for production deployment.

One can use 

PERL5LIB=/home/koha/src KOHA_CONF=/home/koha/etc/koha-conf.xml starman -M FindBin --workers 6 --port 5000 --pid /home/koha/var/run/starman.pid /home/kohaadmin/prod.psgi 2>/home/koha/var/log/starman.err >/home/koha/var/log/starman.log &

And it will launch 6 workers. Suggestion is to use one worker per each server CPU core.

You also might want to use a socket and not listen on a port.

When deploying in production turn off all debugging modules because some of them are known to leak memory. It's also recommended to decrease number of requests to something like --max-requests 50 which will force Starman's child processes to be re-spawn every 50 requests to keep memory usage at sane values (default value for max requests is 1000 which is too much for current Koha code and can make each child very big).

PSGI Application

adapting web frameworks to psgi mode

PSGI specification

How PSGI was introduced in gitweb

24 days of Plack and PSGI tips and tricks.


Minimal 'no thrills, no CPAN' version for Debian Squeeze

To try Plack on Debian Squeeze with package installation only (but also without fancy debugging stuff) you can use this minimal version. Make sure you have squeeze-backports enabled and do 

 sudo apt-get install libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl

Create the following files. For a git installation, replace /srv/koha with /path/to/your/kohaclone. For a regular package installation, uncomment the alternative paths, comment the respective line above. See Sections above for how to start with plackup.

opac.psgi

#!/usr/bin/perl
use Plack::Builder;
use Plack::App::CGIBin;
use Plack::App::Directory;
use lib("/srv/koha");
#use lib("/usr/share/koha/lib");
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;
use C4::Branch;
use C4::Category;

my $app=Plack::App::CGIBin->new(root => "/srv/koha/opac");
#my $app=Plack::App::CGIBin->new(root => "/usr/share/koha/opac/cgi-bin/opac");


builder {
        enable "Plack::Middleware::Static",
                path => qr{^/opac-tmpl/}, root => '/srv/koha/koha-tmpl/';
#                path => qr{^/opac-tmpl/}, root => '/usr/share/koha/opac/htdocs/';

        enable 'StackTrace';
        mount "/cgi-bin/koha" => $app;
};

intranet.psgi

#!/usr/bin/perl
use Plack::Builder;
use Plack::App::CGIBin;
use Plack::App::Directory;
use lib("/srv/koha");
#use lib("/usr/share/koha/lib");
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;
use C4::Branch;
use C4::Category;

my $app=Plack::App::CGIBin->new(root => "/srv/koha/");
#my $app=Plack::App::CGIBin->new(root => "/usr/share/koha/intranet/cgi-bin/");

builder {
        enable "Plack::Middleware::Static",
                path => qr{^/intranet-tmpl/}, root => '/srv/koha/koha-tmpl/';
#                path => qr{^/intranet-tmpl/}, root => '/usr/share/koha/intranet/htdocs/';

        enable 'StackTrace';
        mount "/cgi-bin/koha" => $app;
};

Another option using Starman and nginx

This someone more complex configuration uses starman and nginx for maximum speed (and confusion!). You will need to install Starman, nginx, and the various debug modules discussed above. You will also need to disable or remove Apache so that nginx can bind to port 80.

This configuration assumes a dev-mode install in ~/koha-dev. Note that all ${VARIABLE} placeholders in the following files must be replaced with appropriate absolute values. You will need the following files:

etc/koha-nginx.conf

server {
    server_name ${DOMAIN};
    access_log  ${HOME}/koha-dev/var/log/nginx-opac-access.log;
    root        ${HOME}/kohaclone/koha-tmpl;

    rewrite ^/$ /cgi-bin/koha/opac-main.pl;
    rewrite ^/cgi-bin/koha/(.*)$ /cgi-bin/koha/opac/$1;

    location    / {
        try_files $uri @plack;
    }

    location @plack {
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass  http://unix:${HOME}/koha-dev/var/run/plack.sock;
    }
}

server {
    server_name intra.${DOMAIN};
    access_log  ${HOME}/koha-dev/var/log/nginx-intranet-access.log;
    root        ${HOME}/kohaclone/koha-tmpl;

    rewrite ^/$ /cgi-bin/koha/mainpage.pl;

    location    / {
        try_files $uri @plack;
    }

    location @plack {
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass  http://unix:${HOME}/koha-dev/var/run/plack.sock;
    }
}

bin/koha.psgi

#!/usr/bin/perl
    
BEGIN {
    $ENV{'KOHA_CONF'} = '${HOME}/koha-dev/etc/koha-conf.xml';
}
    
use lib("${HOME}/kohaclone");
use lib("${HOME}/kohaclone/installer");
use lib("${HOME}/perl5");
use Plack::Builder; 
use Plack::App::CGIBin;
use Plack::App::Directory;
use Plack::Middleware::Debug;
use Plack::App::URLMap;
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;   
use C4::Branch;
use C4::Category;         

C4::Context->disable_syspref_cache();

my $koha = Plack::App::CGIBin->new(root => "/home/jcamins/kohaclone");

builder {
    enable 'Debug',  panels => [
        qw(Environment Response Timer Memory),
#               'Profiler::NYTProf', # FIXME this should work, but is missing .gif exclude
#        [ 'Profiler::NYTProf',
#            exclude => [qw(.*\.css .*\.png .*\.ico .*\.js .*\.gif)],
#            base_URL => '/nytprof',
#            root => '/home/jcamins/kohaclone/koha-tmpl/nytprof',
#            minimal => 1 ],
#               [ 'DBITrace', level => 1 ],
        ];

    enable "Plack::Middleware::Static";

    enable 'StackTrace';

    mount '/cgi-bin/koha' => $koha;
};

bin/koha-plack.sh

#!/bin/sh -e
    
# This file is part of Koha.
#   
# Koha is free software; you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the Free Software
# Foundation; either version 3 of the License, or (at your option) any later
# version.
#
# Koha is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with
# Koha; if not, write to the Free Software Foundation, Inc., 59 Temple Place,
# Suite 330, Boston, MA  02111-1307 USA

### BEGIN INIT INFO
# Provides:          koha-plack
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Plack server for Koha
### END INIT INFO

export KOHA_CONF=${HOME}/koha-dev/etc/koha-conf.xml
export OPACDIR=${HOME}/kohaclone/opac
export INTRANETDIR=${HOME}/kohaclone
export LOGDIR=${HOME}/koha-dev/var/log
        
PIDFILE=${HOME}/koha-dev/var/run/plack.pid

# Use one of these lines to choose between TCP port and UNIX socket listeners
SOCKET=${HOME}/koha-dev/var/run/plack.sock
#PORT=5000

case "$1" in
    start)
        opt="$opt --access-log $LOGDIR/koha-access.log --error-log $LOGDIR/koha-error.log"
        opt="$opt -M FindBin --max-requests 50 --workers 2 -E deployment"
        if [ $SOCKET ]; then
            opt="$opt --listen $SOCKET -D --pid $PIDFILE"
        elif [ $PORT ]; then
            opt="$opt --port $PORT -D --pid $PIDFILE"
        fi
        /usr/bin/starman $opt ${HOME}/koha-dev/bin/koha.psgi

        if [ $SOCKET ]; then
            chown ${USER}:www-data $SOCKET
        fi
        ;;
    stop)
        start-stop-daemon --stop --pidfile $PIDFILE
        ;;
    *)
        echo "Usage: koha-plack {start|stop}"
        exit 1
        ;;
esac

Apache with Plack for intranet with site configuration support

This instructions is intended to be used with Koha installations which have configuration files in layout used by koha-common Debian package. This is effort to provide simplest possible migration from Koha package installation to intranet using plack. Advantage is speed increase and minimal manual configuration changes.

However this configuration uses just configuration and log files in layout which is same as koha-common package and doesn't use koha files from installation package, but from git checkout in /srv/koha_ffzg, so it will need some modifications for your particular installation.

/etc/init.d/koha-plack

#!/bin/sh -e

# This file is part of Koha.
#
# Koha is free software; you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the Free Software
# Foundation; either version 3 of the License, or (at your option) any later
# version.
#
# Koha is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with
# Koha; if not, write to the Free Software Foundation, Inc., 59 Temple Place,
# Suite 330, Boston, MA  02111-1307 USA

### BEGIN INIT INFO
# Provides:          koha-plack
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Plack server for Koha
### END INIT INFO

DESC="Koha Plack server"

export SITE=ffzg
export BASE_DIR=/srv/koha_ffzg
export LOGDIR=/var/log/koha/${SITE}/

PIDFILE=/var/run/koha/${SITE}/plack.pid

do_start()
{
        opt="$opt --access-log $LOGDIR/plack-access.log --error-log $LOGDIR/plack-error.log"
        # --max-requests 1 is essential to avoid re-using child with corrupted utf-8
        # --workers 8 might be adjusted to number of cores on system
        opt="$opt -M FindBin --max-requests 1 --workers 8 -E deployment --user ${SITE}-koha"
        opt="$opt --port 5000 --host 127.0.0.1 --pid $PIDFILE -D"

        /usr/bin/starman $opt ${BASE_DIR}/misc/plack/koha-${SITE}.psgi
}

do_stop()
{
        start-stop-daemon --stop --pidfile $PIDFILE
}

case "$1" in
        start)  
                echo "Starting $DESC"
                do_start
                ;;
        stop)   
                echo "Stopping $DESC"
                do_stop
                ;;
        reload) 
                echo "Reloading $DESC"
                start-stop-daemon --stop --signal HUP --pidfile $PIDFILE
                ;;
        restart)
                echo "Restarting $DESC"
                test -f $PIDFILE && do_stop
                do_start
                ;;
        *)
                echo "Usage: $0 {start|stop|restart}"
                exit 1
                ;;
esac

koha-ffzg.psgi

You can use koha.psgi from this page with one important change: add ReverseProxy plack middleware (sadly not in Debian packages) to generate correct URL's in CGI.pm which koha uses. So simplest builder part which works is: 

# please include everything above this from other example on this page!

my $app=Plack::App::CGIBin->new(root => '/srv/koha_ffzg');

builder {
        enable 'ReverseProxy';
        mount "/cgi-bin/koha" => $app;
};

You don't need any other modules since static files are served by apache itself and additional modules create bigger memory footprint for your plack processes.

Apache

Edit your apache site configuration file, e.g. /etc/apache2/sites-enabled/ffzg.conf and insert redirect to plack between apache-shared.conf and apache-shared-intranet.conf like this:

Apache 2.2 in Debian squeeze doesn't support unix socket connection, so this example uses localhost:5000 to run plack server.

This also assumes that your intranet is behind https (which it should be) so we are adding additional header X-FORWARDED-PROTO which will be picked up by ReverseProxy plack middleware to create correct URLs. 

   Include /etc/koha/sites/ffzg/apache-shared.conf

   # redirect to plack
   ProxyPreserveHost On
   RequestHeader set X-FORWARDED-PROTO "https"
   ProxyPass /cgi-bin/koha http://localhost:5000/cgi-bin/koha

   Include /etc/koha/sites/ffzg/apache-shared-intranet.conf

Enable modules required for this configuration and restart apache 

# a2enmod headers proxy_http
Enabling module headers.
Considering dependency proxy for proxy_http:
Module proxy already enabled
Module proxy_http already enabled
Run '/etc/init.d/apache2 restart' to activate new configuration!

test your new plack installation

Create file /srv/koha_ffzg/test.pl 

#!/usr/bin/perl

use CGI;

my $q = new CGI;

print $q->header, $q->self_url;

When accessing this page through URL like https://intranet.example.com/cgi-bin/koha/test.pl you should see same URL in your browser on displayed on page. This ensures that all our effort with reverse proxy was worth it and that your new installation works as expected. You can remove test.pl file after you are sure that installation works.

/etc/logrotate.d/koha-common

starman doesn't support log rotation on reload, so we need to restart plack server to get rotation. insert koha-plack restart in postrotate: 

    postrotate
        /etc/init.d/apache2 reload > /dev/null
        /etc/init.d/koha-plack restart > /dev/null
    endscript



Improving Plack performance

Add the import of Koha::Schema into the intranet.psgi to load the database schema and improve the performance of a checkout.

intranet.psgi

#!/usr/bin/perl
use Plack::Builder;
use Plack::App::CGIBin;
use Plack::Middleware::Debug;
use Plack::App::Directory;
#use Plack::Middleware::Debug::MemLeak;
use lib("/srv/koha");
use C4::Context;
use C4::Languages;
use C4::Members;
use C4::Dates;
use C4::Boolean;
use C4::Letters;
use C4::Koha;
use C4::XSLT;
use C4::Branch;
use C4::Category;
use Koha::Schema;

my $app=Plack::App::CGIBin->new(root => "/srv/koha/");

builder {
        enable 'Debug',  panels => [
                qw(Environment Response Timer Memory),
#               [ 'Profiler::NYTProf', exclude => [qw(.*\.css .*\.png .*\.ico .*\.js .*\.gif)] ],
#               [ 'DBITrace', level => 1 ],
        ];

        enable "Plack::Middleware::Static",
                path => qr{^/intranet-tmpl/}, root => '/srv/koha/koha-tmpl/';

#       enable "Plack::Middleware::Static::Minifier",
#               path => qr{^/intranet-tmpl/},
#               root => './koha-tmpl/';

        enable 'StackTrace';
        mount "/cgi-bin/koha" => $app;

};

Use Devel::NYTProf to see the improvement.

Also have a look on Wiki Performance to activate caching.

Retrieved from "https://wiki.test.koha-community.org/w/index.php?title=Plack&oldid=19883"