developer diaries

Step (1): Create a new package

Go to ACS Site-Wide Administration and it’s Package Manager. At the bottom you can Create a new package

Choose a short readable package key and name. Choose to inherit from templates in order extend on XoWiki. You specifically don’t want to implement subsite semantics (just on a sidenote). Everything else is up to you.

If everything went accordingly, you’ll get a screen listing all the details for your new package.

Next to other packages you will now have a dedicated package in your packages folder of your openacs distribution:

(2) Setup Dependencies and Provides

We want to extend on XoWiki, therefore we need to somehow specify that somewhere. On the summary page of your newly created package scroll down and click Dependencies and Provides

Click “Add a service extended by this package” and select XoWiki.

After setting up the extension you will now be able to build on XoWiki. What does this mean. Basically two things:

Reusability & Extendibility: By extending XoWiki you will benefit from all features already present within the XoWiki package, thus reusing XoWiki technology. On top of that you can extend functionality in order to customize your XoWiki based application or build new features which are not present in XoWiki.

(3) Parameter Page

When looking at the parameter page, you will now be able to use parameters based on the XoWiki application and you can define separate parameters for specifically for your package:

In order to setup custom parameters, go to ACS Site-Wide Administration / Package Manager and click on the name of your package (Xowiki Sample Application). There, you can click on Parameters under the Manage headline:

Simply click “Add new parameter” to specify details for your parameter.

This will result in an entry on the parameter page of XoWiki Sample Application

You can access this parameter via the XoWiki procedure called get_parameter

An example would be

set title [$package_id get_parameter PackageTitle [$package_id instance_name]]

(4) Prototype Pages

Read up on Prototype Pages in the official XoWiki Documentation

Create a prototype directory and index.page file in your XoWiki Sample Application folder (xowiki-sample-app/www/prototypes/index.page). It should contain the following basic setup to get started:

You basically define a starting point for your wiki. A simple index page. If you delete it, it will be created automatically from the prototype defined in your XoWiki Sample Application. The convenience here, is that you can define your own custom index page with your package - this might come in handy when deploying your application and providing a starting point. You can edit this page with your wiki and customize it to your likes. 

(5) Templates

Templates are based on ACS-Templating. The documentation can be found here. 

To get started simply copy the structure from your xowiki folder over to your xowiki-sample-app. You need a template folder located in resources: /xowiki-sample-app/resources/template. Now copy over view-default.adp from xowiki into your empty templates folder. In order to demonstrate functionality remove the title in the template and name the file view-example.adp

Note that I only changed things around at the adp placeholders. The top stuff concerns xowiki’s menu structure, which you can also customize in this case. For example to include or hide additional links for users.

Now all you need to do is specify that you are using view-example.adp as your template file in the parameter page of your sample application. (Default Setting: view-default)

When refreshing your index page from the sample application the title should be gone. (demonstration wise)

With this setup you can start designing your website based on XoWiki in a separate package (xowiki-sample-app).

If you need a fresh slate to work with you can also issue a pull request from this xowiki-sample-app via github after creating a new OpenACS application.

https://github.com/michaelfeurstein/xowiki-sample-app.git

Here’s a working example for a LaunchAgent I use to start NaviServer:

<?xml version=“1.0” encoding=“UTF-8”?> <!DOCTYPE plist PUBLIC “-//Apple Computer//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd”><plist version=“1.0”>   <dict> <key>Label</key> <string>org.naviserver</string> <key>ProgramArguments</key> <array> <string>/usr/local/ns/bin/nsd</string> <string>-f</string> <string>-u</string> <string>nsadmin</string> <string>-g</string> <string>nsadmin</string> <string>-t</string> <string>/usr/local/ns/conf/nsd-config.tcl</string> </array> <key>StandardOutPath</key>           <string>/usr/local/var/log/naviserver_daemon.log</string>       <key>StandardErrorPath</key>           <string>/usr/local/var/log/naviserver_daemon.log</string> </dict> </plist>

The file is saved under /Library/LaunchAgents/org.naviserver.plist

Load the LaunchAgent:

sudo launchctl load org.naviserver.plist

Start the LaunchAgent:

sudo launchctl start org.naviserver

Note the different input values for loading and starting through launchctl. The start command only takes the Label as an input.

Stop the LaunchAgent:

sudo launchctl stop org.naviserver

Unload the LaunchAgent:

sudo launchctl unload org.naviserver.plist

Some useful links:

Launchd Info: http://launchd.info/

System Integrity Protection (SIP) https://support.apple.com/en-us/HT204899

Note on running navisever in foreground mode through LaunchDaemon: http://superuser.com/a/1112872/311059 (Thanks for this one)

Overview of your whole library. Including keywords, folders and paper link.

Edit each entry according to defined fields. The predefined formats are very useful. You can add fields, if needed, by right clicking inside the form.

How I currently use the Annote tab to summarise the most important points

Look up the paper and get the link to the direct BibTeX representation. BibDesk supports several formats from scientific websites. Depending on the format BibDesk can also extract bibliographic information from web pages.

With the copied link you can directly import the BibTeX format without manually adding specific fields or even creating the BibTeX entry. This is the most time saving feature.

The preview function. 

Unfortunately it’s only available for Mac OS X 10.6 and above.

(0) If you don't have the script from part 1 - go grab it

https://github.com/gustafn/install-ns/blob/master/install-oacs.sh

(1) Check your variables before running the script

sudo ./install-oacs.sh

especially check that the following are set correctly

pg_dir=/opt/local/lib/postgresql94/ install_dotlrn=1

somewhere along the lines there is another pg_dir

pg_dir=${pg_dir}

check that psql is pointing to your current psql install

cd /opt/local/bin sudo ln -s /opt/local/lib/postgresql94/bin/psql psql

(2) Be sure to start postgresql94 server

sudo launchctl start org.postgresql.mifeurst

(3) Make sure cvs installed

port search cvs sudo port install cvs

(4) Run the install script

sudo bash ./install-oacs.sh build

(5) Configure your setup

ns_section "ns/db/driver/postgres" ns_param      pgbin   "/opt/local/lib/postgresql94/bin/"

Check that db_host db_port and db_user are set correctly

set address 0.0.0.0

(6) Start naviserver with the openacs config

As already shown in part 2 of this tutorial writing you will need to customize your openacs-5-8-config.tcl file in order to connect to the correct database. Note that the needed oacs-5-8 database was already created via the install script from above. So no need to create one manually.

sudo /usr/local/ns/bin/nsd -t /usr/local/ns/conf/openacs-5-8-config.tcl -u nsadmin -g nsadmin

(7) Per default OpenACS is installed here

/var/www/oacs-5-8/openacs-4

You’ll find logs, and process ids in the log directory

(8) In my case I got an error

Error: Error parsing queryfile /var/www/oacs-5-8/packages/acs-tcl/lib/page-error.xql: invalid command name "dom" invalid command name "dom"    while executing "dom parse -simple [lindex $args 1]"    (procedure "xml_parse" line 4)    invoked from within "xml_parse -persist $stuff_to_parse"    (procedure "db_qd_internal_parse_init" line 4)    invoked from within "db_qd_internal_parse_init $whole_file $file_tag"    (procedure "db_qd_internal_load_queries" line 15)    invoked from within "db_qd_internal_load_queries $stream $file_path"    (procedure "db_qd_internal_load_cache" line 9)    invoked from within "db_qd_internal_load_cache $file_path"

When further inspecting the error log one finds this:

error reading package index file /usr/local/ns/lib/tdom0.8.3/pkgIndex.tcl: expected version number but got "0.8.3"load"

Add a space after 0.8.3 so /usr/local/ns/lib/tdom0.8.3/pkgIndex.tcl looks like this:

package ifneeded tdom 0.8.3 "load [list [file join $dir libtdom0.8.3.dylib]];         source [list [file join $dir tdom.tcl]]"

Now restart your OpenACS instance.

(9) Navigate to localhost:8000

You should see a welcome page prompting for further steps to install openACS.

If you look through your error.log closely you’ll also see:

Notice: Bootstrap: Installation is not complete - sourcing the installer

Follow the installer via the web interface and setup your OpenACS instance.

After a successful installation your database should include some new entries:

(10) Start your server and go places

Index: http://localhost:8000/

Site-Wide Administration: http://localhost:8000/acs-admin/

Package Manager: http://localhost:8000/acs-admin/apm/

Also note that now your error.log will include the following line:

Notice: Bootstrap: Done loading OpenACS

You now have a working installation of OpenACS with NaviServer and PostgreSQL installed. 

Continue by installing Developer Support via the install interface from the repository. Start by going the package manager's installation page:

http://localhost:8000/acs-admin/install/

(11) Read up on the official documentation

You will probably want to start out with a simple package installation either as a front end application or a backend service.

The openACS documentation is pretty massive and in depth - so go ahead and dive in!

Introduction to packages in OpenACS: http://openacs.org/doc/packages.html Package Development Tutorial: http://openacs.org/doc/tutorial-newpackage.html Getting Started: http://openacs.org/storage/view/tutorial%5C/OpenACS_Tutorial.pdf Introduction to the concept of ACS Objects: http://openacs.org/doc/objects.html 

Enjoy!

Appendix A

You will probably want to have several instances of openacs around to test things or maybe have a development, staging, production workflow. In my case, I like to have a personal playground instance (sandbox?) and a development instance for each project to work on. When installing such an instance the following needs to be kept in mind:

1) Install a new instance via the install-oacs.sh script sudo bash ./install-oacs.sh build

2) Move/Rename the fresh installation sudo mv /var/www/openacs-5-9 /var/www/openacs-5-9-myProject

3) Move/Rename the openacs config file sudo cp /usr/local/ns/conf/openacs-config.tcl /usr/local/ns/conf/openacs-myProject-config.tcl

4) Be sure to have the correct configuration set port set server  set db_name

5) Rename the fresh openacs database ALTER DATABASE "oacs-5-9" RENAME TO "openacs-5-9-myProject-db";

================================================== Step (A) Getting to know adp ==================================================

Starting Points: http://naviserver.sourceforge.net/n/manual/files/adp-overview.html NaviServer Built-in Commands: http://naviserver.sourceforge.net/n/naviserver/files/ns_info.html

The basic examples shown here work very well: http://naviserver.sourceforge.net/n/manual/files/adp-overview.html. I extend upon them.

Errata: user-agent needs cases and should read User-Agent in order to work

================================================== Step (B) Create your own helloWorld.adp ==================================================

(1) Fill your database with dummy data

My database looks like this:

(2) Create your adp's

Create a folder named helloworld in the pages directory of your NaviServer installation:

mkdir /usr/local/ns/pages/helloworld

Create the adp named: db.adp

<html> <head> <title>DB Example</title> </head> <body> <p>Select a db table from the default db pool:</p> <form method="POST" action="db2.adp"> <select name="Table"> <% set db [ns_db gethandle] set sql "select * from cities" set row [ns_db select $db $sql] while {[ns_db getrow $db $row]} {        set table [ns_set get $row name]        ns_puts "<option value=\"$table\">$table" } %> </select> <input type="submit" value="Show Data"> </form> </body> </html>

Create a second adp named: db2.adp

<html> <head> <title>DB Example page 2</title> </head> <body> <h1>DB Example page 2</h1> <% set table [ns_set get [ns_conn form] Table] set db [ns_db gethandle] %> Weather data for <%= $table %>: <table border="1"> <% set row [ns_db select $db "select * from weather where city='$table'"] set size [ns_set size $row] while {[ns_db getrow $db $row]} {        ns_puts "<tr>"        for {set i 0} {$i < $size} {incr i} {                ns_puts "<td>[ns_set value $row $i]</td>"        }        ns_puts "</tr>" } %> </table> </body> </html>

If you are having trouble connecting to the database be sure to check on your NaviServer config file and make sure that you have all sections and parameters setup correctly as shown in part 2 of this tutorial writing.

You can closely follow the database interaction in the error.log of your NaviServer installation.

db.adp should look something like this:

producing a valid database selection returning this:

You should now be familiar with the following:

(1) dynamic tcl scripting in an adp page with script <% %> inclusion (2) form handling with adp form action (3) simple database handling

================================================== Step (A) Conf(ront)(igur)ing your LaunchDaemons ==================================================

Some general concerns on designing integrated high quality linux applications.

One could use daemontools. However I did not get it to work on my Mac environment so I chose to stick with what Apple suggests: LaunchDaemon. If you still prefer daemontools maybe this example writeup will help you get started.

(1) Read up on what a LaunchDaemon is

This tutorial on launchd http://launchd.info/ gives a nice overview. Over at openacs.org some NaviServer related example LaunchDaemons already exist. If you still need more information this post is quite informative. Last but not least you can always consult Apple's mega documentation if in doubt.

(2) Create a LaunchDaemon for NaviServer

Navigate to:

/System/Library/LaunchDaemons

Create a file called:

org.naviserver.yourname.plist

Insert the following:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0">    <dict>        <key>Label</key>        <string>org.naviserver.yourname</string>        <key>ProgramArguments</key>        <array>                <string>/usr/local/ns/bin/nsd</string>                <string>-f</string>                <string>-u</string>                <string>nsadmin</string>                <string>-g</string>                <string>nsadmin</string>                <string>-t</string>                <string>/usr/local/ns/conf/nsd-config.tcl</string>        </array>        <key>KeepAlive</key>        <false/>        <key>StandardOutPath</key>        <string>/usr/local/var/log/naviserver_daemon.log</string>        <key>StandardErrorPath</key>        <string>/usr/local/var/log/naviserver_daemon.log</string>    </dict> </plist>

Explanations:

Label is the key to identify this LaunchDaemon = org.naviserver.yourname

ProgramArguments contains an array of strings with the actual command to start NaviServer. Note that you need to add each parameter and value separately. Also, pay attention to add the foreground flag in order for launchd to see the process (backgroundinfo: http://superuser.com/a/1112872/311059) 

I also define a StandardOutPath and StandardErrorPath feeding into /var/log/naviserver.log in order to keep track of dubious Daemon messages.

(3) Work with launchctl to list, load/unload, start/stop NaviServer Note the different input paramaters for load/unload and start/stop.

sudo launchctl list | grep ‘yourname’ sudo launchctl unload org.naviserver.yourname.plist sudo launchctl load org.naviserver.yourname.plist sudo launchctl start org.naviserver.yourname sudo launchctl stop org.naviserver.yourname

================================================== Step (B) Setup your new database environment ==================================================

(0) Where is what

psql binaries such as psql itself, createdb, dropdb etc are located here:

/opt/local/lib/postgresql94/bin/

which psql94 should give you sth like this

/opt/local/bin/psql94

psql94 points to psql in the postgresql94 bin directory

/opt/local/bin/psql94 -> /opt/local/lib/postgresql94/bin/psql

(1) Create a database cluster with initdb

sudo mkdir -p /opt/local/var/db/postgresql94/defaultdb sudo chown postgres:postgres /opt/local/var/db/postgresql94/defaultdb sudo su postgres -c '/opt/local/lib/postgresql94/bin/initdb -D /opt/local/var/db/postgresql94/defaultdb'

or

sudo su postgres /opt/local/lib/postgresql94/bin/initdb -D /opt/local/var/db/postgresql94/defaultdb

(2) Check on the configuration file

sudo vi /opt/local/var/db/postgresql94/defaultdb/postgresql.conf

(3) Turn some things on

autovacuum = on track_counts = on default_with_oids = on

(4) Start the PSQL server manually and ensure functionality

sudo su postgres -c '/opt/local/lib/postgresql94/bin/postgres -D /opt/local/var/db/postgresql94/defaultdb' &

or

sudo su postgres -c '/opt/local/lib/postgresql94/bin/pg_ctl -D /opt/local/var/db/postgresql94/defaultdb -l logfile start' &

(5) Add user postgres to nsadmin group

sudo /usr/sbin/dseditgroup -v -o edit -a postgres -t user nsadmin

(6) Create your database user For ease of use I prefer to create the same user I use for my system.

/opt/local/lib/postgresql94/bin/createuser -U postgres yourUserName /opt/local/lib/postgresql94/bin/createdb -U postgres testdb /opt/local/lib/postgresql94/bin/psql testdb

(7) Setup a LaunchDaemon in /System/Library/LaunchDaemons

Navigate to:

/Library/LaunchDaemons

Create a file called:

org.postgresql.yourname.plist

Insert the following:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0">    <dict>        <key>Label</key>        <string>org.postgresql.mifeurst</string>        <key>UserName</key>        <string>postgres</string>        <key>GroupName</key>        <string>nsadmin</string>        <key>ProgramArguments</key>        <array>                <string>/opt/local/lib/postgresql94/bin/postgres</string>                <string>-D</string>                <string>/opt/local/var/db/postgresql94/defaultdb</string>        </array>        <key>StandardOutPath</key>        <string>/usr/local/var/log/postgresql_daemon.log</string>        <key>StandardErrorPath</key>        <string>/usr/local/var/log/postgresql_daemon.log</string>    </dict> </plist>

(8) Ensure that the var log is writable for the postgres user This can be particularly frustrating as the LaunchDaemon will through the following error in /var/log/system.log: Service could not initialize: 15G31

sudo touch /usr/local/var/log/postgresql_daemon.log sudo chown postgres:postgres /usr/local/var/log/postgresql_daemon.log

(9) Work with launchctl to list, load/unload, start/stop PostgreSQL

sudo launchctl list | grep ‘yourname’ sudo launchctl unload org.postgresql.yourname.plist sudo launchctl load org.postgresql.yourname.plist sudo launchctl start org.postgresql.yourname sudo launchctl stop org.postgresql.yourname

You should now see something like this:

For superuser access login with postgres user (note the hash on the prompt):

psql94 -U postgres testdb

Drop the database if you don’t want it anymore. with drodb.

================================================== Step (C) Configuring NaviServer to work with PostgreSQL ==================================================

For a successful database connection ensure the following setup for your conf.tcl file:

set db_host     localhost set db_port     "5432" set db_user     yourUsername set db_name     testdb

ns_section     "ns/db/drivers" ns_param       postgres           nsdbpg.so

ns_section     "ns/db/pools" ns_param       pool1           "PostgresSQL Database Pool"

ns_section     "ns/db/pool/pool1" ns_param        driver              postgres                           ns_param        connections         64                                 ns_param        user                ${db_user}                         ns_param        password            ""     ns_param        datasource          ${db_host}:${db_port}:${db_name}   ns_param        pgbin               /opt/local/lib/postgresql94/bin/   ns_param        verbose             On     ns_param        logsqlerrors        On     ns_param        extendedtableinfo   On                                 ns_param        maxidle             31536000 ns_param        maxopen             31536000

ns_section     "ns/server/default/db" ns_param        pools               *       ns_param        defaultpool         pool1

================================================== Step (A) Installing NaviServer ==================================================

(1) install xcode

do so by simply downloading and installing via appstore be sure to open xcode and accept the terms

(2) install xcode command line tools (yosemite)

sudo xcode-select —install

(3) accept the xcode licence via terminal

sudo xcodebuild -license

(4) install MacPorts

https://distfiles.macports.org/MacPorts/MacPorts-2.3.3-10.10-Yosemite.pkg

(5) update MacPorts

sudo port selfupdate sudo port upgrade outdated

(6) install postgresql94 Note: postgresql93 did not work for me on mac os x yosemite

sudo port install postgresql94 sudo port install postgresql94-server

(7) git clone install-ns scripts

Note: create directories as needed if missing

/usr/local/bin /usr/local/include /usr/local/lib /usr/local/ns /usr/local/src cd /usr/local/src git clone https://github.com/gustafn/install-ns.git cd install-ns

(8) chmod

chmod 755 install-ns.sh chmod 755 install-oacs.sh

(9) create user nsadmin

do it via system preferences on mac

or do so via terminal (in local / domain)

dscl / -create /Users/yourusername dscl / -create /Users/yourusername UserShell /bin/bash dscl / -create /Users/yourusername RealName “nsadmin” dscl / -create /Users/yourusername UniqueID 503 dscl / -create /Users/yourusername PrimaryGroupID 1000 dscl / -create /Users/yourusernameNFSHomeDirectory/Local/Users/nsadmin passwd nsadmin

(10) create group nsadmin add nsadmin to group nsadmin

/usr/sbin/dseditgroup -v -o create nsadmin /usr/sbin/dseditgroup -v -o edit -a nsadmin -t user nsadmin

(11) be sure to have wget installed

sudo port search wget sudo port install wget

(12) run the install script

sudo bash ./install-ns.sh build

(13) start server

sudo /usr/local/ns/bin/nsd -f -u nsadmin -g nsadmin -t /usr/local/ns/conf/nsd-config.tcl

Note: the pid is written to /usr/local/ns/nsd.pid Or simply use process status

ps -A | grep nsd

(14) check if the server is running

Navigate to localhost:8080 You should see the default welcome page for NaviServer

================================================== Step (B) Installing nsstats and nsconf ==================================================

Do so by clicking the links on the index page of your naviserver installation. You should get a response if the installation was successful. If a blank page stares at you consult the error.log (/usr/local/ns/logs/)

In my case the error.log reported:

[12/Feb/2015:17:37:55][720.1053de000][-conn:default:1] Error: GET /bitbucket-install.tcl, PeerAddress: 137.208.114.129 couldn't execute "wget": no such file or directory while executing "exec wget --quiet --no-check-certificate -O /tmp/nsstats.tar.gz $source"

Hence i updated the bitbucket script with the exact location of wget:

/opt/local/bin/wget

Follow the instructions on setting a password in the nsconf.tcl file.

You now should see this:

================================================== Step (C) Read up on NaviServer ==================================================

Here: http://naviserver.sourceforge.net/n/manual/files/admin-maintenance.html#section2

Help: