release 0.0 · 1.we do not have sqllite working [#]_ as a backend so we must have netowrked...

rhaptos2 DocumentationRelease 0.0.1

Paul Brian

February 25, 2014

Contents

1 Testing 31.1 To test locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 config passing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 footnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Config 52.1 Future notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Session Management 7

4 Summary 94.1 What is a session? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.3 Authentication flow and sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.4 Difference between session sign on and Single Sign On . . . . . . . . . . . . . . . . . 10

5 Known issues 115.1 What is wrong with current setup? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.2 What about API Tokens? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.3 Testing issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.4 Why do you not encrypt the session ID in the cookie? . . . . . . . . . . . . . . . . . . . . . . . . . 12

6 Documenting JSON flow of repo API 136.1 POST /module/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136.2 PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . . 136.3 PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . . 146.4 POST /folder/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156.5 PUT /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . . . . 156.6 GET /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . . . . 166.7 POST /collection/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166.8 PUT /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 . . . . . . . . . . . . . . . 176.9 GET /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 . . . . . . . . . . . . . . . 186.10 PUT /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 . . . . . . . . . . . . . . . 186.11 PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . . 196.12 PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . . 206.13 PUT /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . . . . 206.14 GET /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . 216.15 GET /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . . . . 21

i

6.16 GET /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . . . 216.17 GET /workspace/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216.18 DELETE /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . 226.19 DELETE /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 . . . . . . . . . . . . . . . 226.20 DELETE /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 . . . . . . . . . . . . . 226.21 DELETE /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 . . . . . . . . . . . . . 236.22 DELETE /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . 236.23 DELETE /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 . . . . . . . . . . . . . . . . . 23

7 Simple spec links 257.1 URIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257.2 User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257.3 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.4 Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.5 Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.6 Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

8 Proposals 278.1 Add Google Analytics support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278.2 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278.3 hooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288.4 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

9 API 299.1 API for Views and models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299.2 Authentication API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319.3 Common functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399.4 rhaptos2.repo.backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.5 rhaptos2.repo.configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.6 rhaptos2.repo.err . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.7 rhaptos2.repo.log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.8 run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.9 rhaptos2.repo.run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

10 Misc. 4310.1 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Python Module Index 45

ii

rhaptos2 Documentation, Release 0.0.1

Welcome to the docs for Rhaptos2.repo.

Some documentation is hand-written as below, and some is API further below

Contents 1


2 Contents

CHAPTER 1

Testing

We have a WSGI app to be tested.

I am using 3 approaches, which more or less marry up.

1. Doctests, and examples.

2. nosetesting for webtest

3. config passing for testing...

• standalone - should run with just python/rhaptos packages

• with network

• with selenium

1.1 To test locally

1. We do not have sqllite working [#]_ as a backend so we must have netowrked postgres working,

2. apart from that run the tests as follows

3. doctests: each file individually should support testmod doctests can run in their own suite - they should notrequire netowrk access. This is not always true.

4. functional testing of wsgi app:

nosetests –tc-file=../../testing.ini runtests.py

This will use webtest to send requests in-process to the app - no http calls are made. The app however does not knowthis and proceeds as if runing in a wsgi server.

5. functional HTTP testing of wsgi app:

nosetests --tc-file=../../pbrian.ini --tc=HTTPPROXY:http://localhost:8000/ runtests.pypython run.py --config=../../pbrian.in

5. example.txt - a demo / example of how the various bits fir together. it is a doctest suite but needs integratedsystem

1.2 config passing

nose

3


1.3 Also

Use interlude During development, copy the below into a doctest:

>>> import interlude; interlude.interact(locals())

now run the doctest, and it drops you into a shell at that line Just use the shell to develop, check, test etc. Its usinglocals() in the doctest. Its really helpful.

biblio

http://ivory.idyll.org/articles/nose-intro.html

1.4 footnotes

4 Chapter 1. Testing

http://bluedynamics.com/articles/jens/interlude-write-python-doctests-interactive

http://ivory.idyll.org/articles/nose-intro.html

CHAPTER 2

Config

After the great configuration debates I am reluctant to touch this too much but nosetests has rather forced our hands.

noestests is a useful piece of kit but has one major problem - its really awkward to pass configuration into to test. DougHellman has a generally accepted soklution - nose-testconfig.

THis effectively takes the config file given on command line and parses it into a python dict and presents it as a variablenamed “config”:

from testconfig import configmain_server = config[’foo’][’bar’]

The current Configuration setup uses the convention of moving all keys under the [app] section into the top level ofthe dict and then everything else lives under “globals”. It returns a Mapping object that acts like a dict. Pretty quicklythis is read off into the app.config object which also acts like a dict.

To satisfy this I muck around with nosetest ini file I find this unsatisfaotry but survieable for now - ultimately I thinkjust producing a dict out of a ini file and being done with it will work. God knows how I drove to a different conclusionat any time.

Ultimately its pretty hacky so there may wel lbe a break somewhere down the line.

2.1 Future notes

nose-testconfig supports YAML and python files so is not particularly limiting, and we can change this again - I didnot particularly want to as it seemed churlish rehashing old stuff but its become a RRPIA.

5


6 Chapter 2. Config

CHAPTER 3

Session Management

7


8 Chapter 3. Session Management

CHAPTER 4

Summary

Session management in the repo has been poor for a long time, and it has made testing of the various functionalitiesmore awkward and requiring more brain use by developer and test suite than needed.

There is a branch (session-cookie-approach) that hopefully fixes this.

We shall provide sensible, secure session management (to decide if a browser has previously authenticated) and linkedto that flow-control that makes decisions based on the session management such as presenting a login screen, a regis-tration screen or whatever we need.

4.1 What is a session?

A session here is a defined period of time in which a CNX server will accept a random number string as a password-replacement/proxy for each HTTP request received.

4.2 Overview

The session shall be activated after a registered user logs in to the repo (via openid) and the session will be assigneda UUID, which shall be stored on the users browser as a cookie, and also cached in a postgres dbase where the usersdetails will be the value to the UUID key.

Every time the user represents their cookie we shall look up the user details in the cache, take any appropriate flow-control action (session timed out ? relogin?) and then store the user details in the request for later use.

Issues such as deleting sessions, retrieving the correct dicts etc are also handled, with some known issues (see below).

4.2.1 Sessions

Please see sessioncache for details.

4.3 Authentication flow and sessions.

Sessions are just a convenient way of allowing a user to signon once only. The flow of authentication is a little moreconvulited

please see rhaptos2.repo.auth.handle_user_authentication()

9


4.4 Difference between session sign on and Single Sign On

The repo manages its own login (openid) and sessions (sessioncache) The user will sign in once and then be given asessionid. We will lookup the user from the the id as long as the session is valid.

If another service (transformations) receives a request from the user how should we validate the user request -should the session cookie be tested against the repo session cache locally?

What if the repo calls the transformation serivce to act on behalf of the user - what should we send from repo totransformations - the sessionid? Another api token created alongside sseession? Where does the lookup occur.

THere are two main phases

• validate already set-up sessions and proceed correctly

• Create and destroy sessions, existing or none (ie login and out)

10 Chapter 4. Summary

CHAPTER 5

Known issues

• I am not handling the situation of user signing in twice.

• I am not handling registraftion (co-ordinate with michael)

• I am not setting cookie expires ...

• I am setting httponly

• ‘SSL’ * 443

5.1 What is wrong with current setup?

1. NO session cache, which was to be redis but never came in. We are storing the users OpenID identifier. This isa massive security hole.

2. reliance on Flask security session implementation. THere are a number of reasons to be disatissfied with this,the first is the secret key is a single phrase, in config.

3. No clear migration away from Flask.

4. The awful temptation to put more and more stuff in session cookies for “ease” and “scalbility”.

Primarily I am frustrated in testing ACLs, and in creating /resources/ - whoch would be again reliant on a brokensession implementation.

5.2 What about API Tokens?

Did we not discuss these at the Sprint? Yes. “Single Sign On” is better decribed as “Once Only Sign On, manysystems” A session is a once-only sign on for a single (local) system. We shall need to have an alternative API tokenapproach for other systems taht want to use the same sign on as authentication. Examples wanted.

5.3 Testing issues

• Creation of a “fake-login-API”. During testing only (ie a flag set) we can visit a API page, and get a valid sessioncookie for one of a number of pre-defined users.

This now exists as /autosession.

11


5.4 Why do you not encrypt the session ID in the cookie?

Mostly because I know bupkiss about encryption. No really I can do AES with OpenSSH just fine, but did I do itright? Did I rotate my encryption keys with each user? Did I use cyclic or block level encryption? Which one is whichagain? Am I handing out an oracle? (The last one is yes)

Here is a simple argument - I contend that to correctly and securely encrypt anything sent client side, when any oneclient could be an attacker, one should have a salt/key unique to each user.

This simple and reasonable request destroys the main argument for sticking session details like isAdmin and UserNameinto a encrypted cookie - that it simplifies distributed architecture (I can let client connect to any web server, and I willstill have the session state in the cookie, no need for a database lookup)

Well the minute we need to get a unique salt for a user, we are back to database lookups, and just as frequently assession lookups.

Anyway, enough round the houses, I don’t know enough about securing encrypted services with part of the serviceunder complete control of the attacker, to be sure I have not screwed it up. So I wont do it till I do, and even then allwe should store is the session ID.

5.4.1 A neat trick

Sometimes it is desireable to set a cookie in your browser - chrome enables us to do this as follows:

1. navigate to the domain & path desired (i.e. “/” in most cases)

2. enter javascript:document.cookie="name=value" in the address bar & return

3. you should then revisit the domain, and hey presto you have a cookie

Thanks to http://blog.nategood.com/quickly-add-and-edit-cookies-in-chrome

12 Chapter 5. Known issues

http://blog.nategood.com/quickly-add-and-edit-cookies-in-chrome

CHAPTER 6

Documenting JSON flow of repo API

THe below is the output of restrest.py. It documents HTTP conversations as they occur through the python re-questsmodule.

6.1 POST /module/

Cookie: cnxsessionid=00000000-0000-0000-0000-000000000000Host: 127.0.0.1:8000Content-Length: 826Content-Type: application/json; charset=utf-8

Body:

{"authors": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": "<h1>In CONGRESS, July 4, 1776.</h1>\n<p>The unanimous Declaration ..."copyrightHolders": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"id_": "cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126","maintainers": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"title": "Introduction"

}

Response:

Content-Type: application/json; charset=utf-8Content-Length: 1131

{"body": "<h1>In CONGRESS, July 4, 1776.</h1>\n<p>The u...

6.2 PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126

NB - we are using session 000 for the initial PUT

13



Body:

{"acl": [

"cnxuser:75e06194-baee-4395-8e1a-566b656f6921"],

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This is the useruri for a registered user, (ross)who we happen to know has a sessionID of 0001

"authors": ["cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"

],"body": "<p> Shortened body in test_put_module","copyrightHolders": [



}

Response:


{"body": "<p> Shortened body in test_put_module", "id_"...


Here we are using a different user, 001 (ross) that we added as a ACL in previous PUT.


Body:

{"acl": [

"cnxuser:75e06194-baee-4395-8e1a-566b656f6921"],"authors": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],

14 Chapter 6. Documenting JSON flow of repo API


"body": "<p> OTHERUSERSESSIONID has set this","copyrightHolders": [



}

Response:


{"body": "<p> OTHERUSERSESSIONID has set this", "id_": ...

6.4 POST /folder/


Body:

{"body": [

"cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126","cnxmodule:350f7859-e6e7-11e1-928f-2c768ae4951b","cnxmodule:4ba18842-1bf8-485b-a6c3-f6e15dd762f6","cnxmodule:77a45e48-6e91-4814-9cca-0f28348a4aae","cnxmodule:e0c3cfeb-f2f2-41a0-8c3b-665d79b09389","cnxmodule:c0b149ec-8dd3-4978-9913-ac87c2770de8"

],"id_": "cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707","title": "Declaration Folder"

}

Response:


{"body": [{"mediaType": "application/vnd.org.cnx.module...

6.5 PUT /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707


6.4. POST /folder/ 15


Body:

{"acl": [

"00000000-0000-0000-0000-000000000001"],"body": [

"cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126","cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41127"

],"id_": "cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707","title": "Declaration Folder"

}

Response:



6.6 GET /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707

Cookie: cnxsessionid=00000000-0000-0000-0000-000000000000Host: 127.0.0.1:8000

Response:

Content-Type: application/json; charset=utf-8Content-Length: 481Access-Control-Allow-Origin: *


6.7 POST /collection/


Body:

{"authors": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": "<ul><li><a href=\"cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126\"..."copyrightHolders": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"id_": "cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7","keywords": [

"Life","Liberty",



"Happiness"],"language": "en","maintainers": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"subType": "Other Report","subjects": [

"Social Sciences"],"summary": "No.","title": "United States Declaration Of Independance"

}

Response:


{"body": "<ul><li><a href=\"cnxmodule:d3911c28-2a9e-415...

6.8 PUT /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7


Body:

{"acl": [

"00000000-0000-0000-0000-000000000001"],"authors": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": "<ul><li><a href=\"cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126\"..."copyrightHolders": [


"Life","Liberty","Happiness"

],"language": "en","maintainers": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"subType": "Other Report","subjects": [

"Social Sciences"],

6.8. PUT /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 17


"summary": "No.","title": "United States Declaration Of Independance"

}

Response:



6.9 GET /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7


Response:



6.10 PUT /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7


Body:

{"authors": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": [

"cnxmodule:SHOULDNEVERHITDB0"],"copyrightHolders": [


"Life","Liberty","Happiness"

],"language": "en","maintainers": [

"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"



],"subType": "Other Report","subjects": [

"Social Sciences"],"summary": "No.","title": "United States Declaration Of Independance"

}

Response:

Content-Type: text/htmlContent-Length: 227

null



Body:

{"acl": [


"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": "Declaration test text","copyrightHolders": [



}

Response:


{"body": "Declaration test text", "id_": "cnxmodule:d39...

6.11. PUT /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 19




Body:

{"acl": [


"cnxuser:f9647df6-cc6e-4885-9b53-254aa55a3383"],"body": "NEVER HIT DB","copyrightHolders": [



}

Response:


null

6.13 PUT /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707


Body:

{"acl": [

"00000000-0000-0000-0000-000000000001"],"body": [

"THIS IS TEST"],"id_": "cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707","title": "Declaration Folder"

}

Response:




null

6.14 GET /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126


Response:


{"body": "Declaration test text", "id_": "cnxmodule:d39...

6.15 GET /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707


Response:

Content-Type: application/json; charset=utf-8Content-Length: 481Access-Control-Allow-Origin: *


6.16 GET /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126


Response:


null

6.17 GET /workspace/


6.14. GET /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 21


Response:

Content-Type: application/json; charset=utf-8Content-Length: 433Access-Control-Allow-Origin: *Access-Control-Allow-Credentials: true

[{"mediaType": "application/vnd.org.cnx.module", "id": ...

6.18 DELETE /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126


Response:


null

6.19 DELETE /module/cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126


Response:


cnxmodule:d3911c28-2a9e-4153-9546-f71d83e41126 is no mo...

6.20 DELETE /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7


Response:


null



6.21 DELETE /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7


Response:


cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 is n...

6.22 DELETE /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707


Response:


null

6.23 DELETE /folder/cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707


Response:


cnxfolder:c192bcaf-669a-44c5-b799-96ae00ef4707 is no mo...

6.21. DELETE /collection/cnxcollection:be7790d1-9ee4-4b25-be84-30b7208f5db7 23

CHAPTER 7

Simple spec links

This is a brief discussion and a linkage to more details of the API spec for CNX rewrite.

7.1 URIs

I have made a mistake here and wish to correct it soon. A URI is a URN, so I used the urh: format to define a URI. Italso tried to avoid any unusual encoding around slashes and CGI escaping. This is confusing and simply not simple.

current string that represents a pure single identifer for a resource:

cnxuser:75e06194-baee-4395-8e1a-566b656f6920

THe new better format:

/user/75e06194-baee-4395-8e1a-566b656f6920

or possibly

/cnxuser/75e06194-baee-4395-8e1a-566b656f6920

If we are versioning the textual changes of say a module:

/module/75e06194-baee-4395-8e1a-566b656f6920@aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d

(sha1hash)

http://www.ietf.org/rfc/rfc4122.txt

7.2 User

The current dict-format for a user

{u’affiliationinstitution’: None,u’affiliationinstitution_url’: None,u’biography’: None,u’email’: None,u’firstname’: None,u’fullname’: u’Paul Brian’,u’homepage’: None,u’identifiers’: [{u’identifierstring’: u’https://paulbrian.myopenid.com’,

u’identifiertype’: u’openid’,

25

http://www.ietf.org/rfc/rfc4122.txt


u’user_id’: u’cnxuser:75e06194-baee-4395-8e1a-566b656f6920’},{u’identifierstring’: u’https://paulbrian.myopenid.com/’,u’identifiertype’: u’openid’,u’user_id’: u’cnxuser:75e06194-baee-4395-8e1a-566b656f6920’}],

u’imageurl’: None,u’interests’: None,u’lastname’: None,u’location’: None,u’middlename’: None,u’otherlangs’: None,u’preferredlang’: None,u’recommendations’: None,u’suffix’: None,u’title’: None,u’user_id’: u’cnxuser:75e06194-baee-4395-8e1a-566b656f6920’,u’version’: None}

7.3 Resources

7.4 Module

7.5 Collection

7.6 Folder

26 Chapter 7. Simple spec links

CHAPTER 8

Proposals

This is an experimental area of the docs - it contains written use cases, specs and just plain old discussions about thework this branch is trying to perform. It seems useful to have one location to put this stuff. This location may or maynot be best.

8.1 Add Google Analytics support

The author(s) of modules and collections should be able to add their own tracking codes to a module or collection orboth, and we should offer a facility to do this.

storyref https://trello.com/card/repo-add-api-support-for-new-metadata-fields-4-pts/5181197901c3b1290b001951/86

8.1.1 Spec

We shall provide a single text field in both modules and collections named googleTrackingID and this will allowarbitrary tracking code to be installed.

The backend repo only needs to support accepting a new field from the json doc and handling it correctly.The ATC client will need to do more, see story https://trello.com/card/atc-add-missing-ui-fields-for-metadata-6-pts/5181197901c3b1290b001951/85

Security issue: It may be better to implement this as a google-only field, and capture only a string corresponding tothe google tracking code (ie AM-1234ABB) and we fill in the script boiler plate around it. This will prevent arbitraryscript being written into the modules. An skype discussion indicated we would sanitise all HTML inputs during thepublication process.

8.1.2 Tests

• Can we inject a arbitrary string?

• Can we see same string returned with no HTML mangling?

8.2 Logging

Good idea. Move to getLogger(__name__)

27

https://trello.com/card/repo-add-api-support-for-new-metadata-fields-4-pts/5181197901c3b1290b001951/86

https://trello.com/card/repo-add-api-support-for-new-metadata-fields-4-pts/5181197901c3b1290b001951/86

https://trello.com/card/atc-add-missing-ui-fields-for-metadata-6-pts/5181197901c3b1290b001951/85

https://trello.com/card/atc-add-missing-ui-fields-for-metadata-6-pts/5181197901c3b1290b001951/85


8.3 hooks

I think we really can do with simple hooks process where we run different functions based on point in request process.

8.4 Logging

We want to uset the getLogger(_name__) form We want to create a context and put it in g / environ and log with thatWe want to clear up logging and redirect to syslog We want to direct syslog elsewhere...

28 Chapter 8. Proposals

CHAPTER 9

API

These documents come directly from source code as API docs (ala epydoc), but are divided up and commented inreasonable order below.

9.1 API for Views and models

9.1.1 Summary

If we move to greenlets as well we need to test that ability. As such I have not introduced a pool for psyocpg2 workyet. (see sessioncache)

9.1.2 API

9.1.3 rhaptos2.repo.views

views.py - View code for the repository application.

Structure: We have three main view-areas.

1. the models (Folder, Collection, Module)

2. the helper views (workspace)

3. binary uploads.

4. openid and persona

Protocols

I try to stick to these

1. Every action (GET POST PUT DELETE) must have a useruri passed in to authorise

2. views recevie back either a model.<> object or a json-encodeable version of that

json-encoding

todo: convert to factory based app entirely todo: remove view / as thats now JS todo: remove apply_cors and applyinternally. Or just use it? todo: remove crash and burn

29


rhaptos2.repo.views.accept_resource_upload()Handler for resource file uploads

rhaptos2.repo.views.apply_cors(resp_as_pytype)A callable function (not decorator) to take the output of a app_end and convert it to a Flask response withappropriate Json-ified wrappings.

rhaptos2.repo.views.auto_session()strictly for testing purposes I want to fake three sessions with known ids. Also generate a “real” session with aknown user FIXME - there has to be a better way

rhaptos2.repo.views.bootstrap()At this point there is either a valid session (so redirect to atc) or there is a need to let the visitor choose either toget an anonymous session, or that they are registered, and they should choose to log in again.

There is a logic choice that might improve things - if they have previously visited us, redirect to /login.

rhaptos2.repo.views.content_router(uid)We now serve everything form api/content

uid = content/1234-1234-12334 ^^^ uuid

router logic is subtly different

1.if we are GET, DELETE, HEAD then no payload and an uid do not collect payload, do collect uid route

2.POST payload no uid

3.PUT payload and uid

(Ignore OPTIONS etc)

rhaptos2.repo.views.folder_router(folderuri)

rhaptos2.repo.views.get_resource(hash)Respond with a the resource file.

rhaptos2.repo.views.index()Serves up the index.html file. This will be removed.

rhaptos2.repo.views.keywords()Returns a list of keywords for the authenticated user.

rhaptos2.repo.views.obtain_payload(werkzeug_request_obj)

rhaptos2.repo.views.requestid()before_request is supplied with this to run before each __call_

rhaptos2.repo.views.simple_xss_validation(html_fragment)

>>> simple_xss_validation("US-12345678-1")True>>> simple_xss_validation("<script>Evil</script>")False

This is very quick and dirty, and we need some consideration over XSS escaping. FIXME

rhaptos2.repo.views.temp_session()When a user wants to edit anonymously, they need to hit this first. This is to avoid the logic problems in knowingif a user should be redirected if they have one but not two cookies etc.

Here we generate a temperoiary userid (that is not linked to cnx-user) then setup a session based on that userid.All work will be lost at end of session.

30 Chapter 9. API


rhaptos2.repo.views.validate_googleTrackingID(payload)Given a (json) formatted payload, return whether the google tracking ID is valid

rhaptos2.repo.views.validate_mediaType(payload)Given a (json) formatted payload, find out if it is a module. collection, folder and return appropriate mediatype

possible enhancements include using a acceptHeader to determine mediatype returns mediatype - seems odd..

rhaptos2.repo.views.verify_schema(model_dict, mediatype)Given a json object, verify it matches the claimed mediaType schema

model_dict: dict of the model as out of json - MUST be pure mediaType, not SOFT form mediatype: WHat wethink the dict confirms to

FixMe: we do not have versioning of schemas FixMe: we don’t have a jsonschema verifier...

rhaptos2.repo.views.versionGET()

rhaptos2.repo.views.whoamiGET()returns Either 401 if OpenID not available or JSON document of form

{“openid_url”: “https://www.google.com/accounts/o8/id?id=AItOawlWRa8JTK7NyaAvAC4KrGaZik80gsKfe2U”, # noqa“email”: “Not Implemented”, “name”: “Not Implemented”}

rhaptos2.repo.views.workspaceGET()

9.2 Authentication API

9.2.1 Summary

We make use of session cookies, a local cache and some logic flow to ensure users can work easily with minimalinterruptions

9.2.2 rhaptos2.repo.auth

How does Authentication, Authorisation and Sessions work?

We are operating a Single-Sign-On service, using Valruse to do the authentication.

Workflow

A user will hit the repo-home page, and :func:‘handle_user_authentication‘will be fired, and based on the cookiesstored in the user browser we will know one of three things about the user.

• Never seen before

• Known user, no in session

• Known user, in session

• Edge cases

Usually they will choose to login and will be directed to the login page on cnx-user. Here, the cnx-user will authenticatethem in some fashion (OpenID) and the redirect the user browser back to the repo, with a token.

The repo then looks up this token against the cnx-user service. And hey presto, if the token matches, the repo knowsthey can trust the browser. (assuming SSL all the way)

9.2. Authentication API 31

https://www.google.com/accounts/o8/id?id=AItOawlWRa8JTK7NyaAvAC4KrGaZik80gsKfe2U


The redirect hits at the /valid endpoint in the repo, which will check the token against the cnx-user, and then

user_uuid_to_user_detailsgiven a authenticated user ID (OpenID), look up the user detailson cnx-user

create_session()

At this point, we now have a user who logged in against cnx-user, has then proven to cnx-repo that they did log in, andnow has a session cookie set in their browser that the repo can trust as password-replacement for a set period.

Temporary sessions

Temporary sessions, or anonymous editing is where a user does not login but uses the repo anonymously, perhapscreating test modules.

This is supported by hitting the endpoint /tempsession which will trigger the view temp_session. This in turn callsset_temp_session(). Here we create both a random sessionID (as per usual for sessions) and we also create arandom user_id. This user_id is used exactly as if it were a real registered user, but it is never sent back to the cnx-user,instead it solely is used in ACLs on the unpub repo.

This way, at the end of a temp session, the user effectively loses all their edits. This may want to be avoided, and ispossible but not yet implemented.

known issues

• requesting_user_id This is passed around a lot This is suboptimal, and I think should be replaced with passingaround the environ dict as a means of linking functions with the request calling them

• I am still passing around the userd in g. This is fairly silly but seems consistent for flask. Will need rethink.

• secure (https) - desired future toggle

• further notes at http://executableopinions.mikadosoftware.com/en/latest/labs/webtest-cookie/cookie_testing.html

rhaptos2.repo.auth.create_session(userdata)A closure function that is stored and called at end of response, allowing us to set a cookie, with correct uuid,before response obj has been created (before request is processed !)

Param userdata - a userdict format.

Returns sessionid

cookie settings:

•cnxsessionid - a fixed key string that is constant

•expires - we want a cookie that will live even if user

shutsdown browser. However do not live forever ...? * httponly - prevent easy CSRF, however allow AJAX torequest browser to send cookie.

rhaptos2.repo.auth.delete_session(sessionid)request browser temove cookie from client, remove from session-cache dbase.

rhaptos2.repo.auth.handle_user_authentication(flask_request)Correctly perform all authentication workflows

We have 16 options for different user states such as IsLoggedIn, NotRegistered. The states are listed below.

THis function is where eventually all 16 will be handled. For the moment only a limited number are.

32 Chapter 9. API

http://executableopinions.mikadosoftware.com/en/latest/labs/webtest-cookie/cookie_testing.html

http://executableopinions.mikadosoftware.com/en/latest/labs/webtest-cookie/cookie_testing.html


Parameters flask_request – request object of pococo flavour.

Returns No return is good because it allows the onward rpocessing of requests.

Otherwise we return a login page.

This gets called on before_request (which is after processing of HTTP headers but before __call__ onwsgi.)

Note: All the functions in sessioncache, and auth, should be called from here (possibly in a chain) and raiseerrors or other signals to allow this function to take action, not to presume on some action (like a redirect)themselves. (todo-later: such late decisions are well suited for deferred callbacks)

Auth Reg InSession ProfileCookie Next Action / RoleType Handled HereY Y Y Y Go YY Y Y N set_profile_cookie YY Y N Y set_session YY Y N N FirstTimeOKY N Y Y ErrorAY N Y N ErrorBY N N Y ErrorCY N N N NeedToRegisterN N Y Y AnonymousGoN N Y N set_profile_cookieN N N Y LongTimeNoSeeN N N N FreshMeatN Y Y Y Conflict with anonymous and reg?N Y Y N Err-SetProfile-AskForLoginN Y N Y NotArrivedYetN Y N N CouldBeAnyone

All the final 4 are problematic because if the user has not authorised how do we know they are registered? Trustthe profile cookie?

we examine the request, find session cookie, register any logged in user, or redirect to login pages

rhaptos2.repo.auth.login()Redirect to cnx-user login.

rhaptos2.repo.auth.logout()kill the session in cache, remove the cookie from client

rhaptos2.repo.auth.lookup_session(sessid)As this will be called on every request and is a network lookup we should storngly look into redis-style lcoaldisk cacheing performance monitoring of request life cycle?

returns python dict of user_details format. or None if no session ID in cache or Error if lookup failedfor other reason.

rhaptos2.repo.auth.session_to_user(flask_request_cookiedict, flask_request_environ)Given a request environment and cookie, return the user data.

>>> cookies = {"cnxsessionid": "00000000-0000-0000-0000-000000000000",}>>> env = {}>>> userd = session_to_user(cookies, env)>>> outenv["fullname"]’pbrian’

Params flask_request_cookiedict the cookiejar sent over as a dict(-like obj).



Params flask_request_environ a dict like object representing WSGI environ

Returns Err if lookup fails, userdict if not

rhaptos2.repo.auth.set_autosession()This is a convenience function for development It should fail in production

rhaptos2.repo.auth.set_temp_session()A temopriary session is not yet fully implemented A temporary session is to allow a unregistered and unautho-rised user to vist the site, acquire a temporary userid and a normal session.

Then they will be able to work as normal, the workspace and acls set to the just invented temporary id.

However work saved will be irrecoverable after session expires...

NB - we have “made up” a user_id and uri. It is not registered in cnx-user. This may cause problems withdistributed cacheing unless we share session-caches.

rhaptos2.repo.auth.store_userdata_in_request(user_details, sessionid)given a userdict, keep it in the request cycle for later reference. Best practise here will depend on web framework.

rhaptos2.repo.auth.user_uuid_to_user_details(ai)Given a user_id from cnx-user create a user_detail dict.

Parameters ai – authenticated identifier. This used to be openID URL, now we directly get backthe common user ID from the user serv.ce

user_details no longer holds any user meta data aparrt from the user UUID.

rhaptos2.repo.auth.user_uuid_to_valid_session(uid)Given a single UUID set up a session and return a user_details dict

Several different functions need this series of steps so it is encapsulated here.

rhaptos2.repo.auth.valid()cnx-user /valid view for capturing valid authentication requests.

rhaptos2.repo.auth.whoami()based on session cookie returns userd dict of user details, equivalent to mediatype from service / session

9.2.3 rhaptos2.repo.sessioncache

sessioncache is a standalone module providing the ability to control persistent-session client cookies and profile-cookies.

sessioncache.py is a “low-level” piece, and is expected to be used in conjunction with lower-level authenticationsystems such as OpenID and with “higher-level” authorisation systems such as the flow-control in auth.py

persistent-session This is the period of time during which a web server will accept a id-number presented as partof an HTTP request as a replacement for an actual valid form of authentication. (we remember that someoneauthenticated a while ago, and assume no-one is able to impersonate them in the intervening time period)

persistent-session cookie This is a cookie set on a client browser that stores a id number pertaining to a persistant-session. It will last beyond a browser shutdown, and is expected to be sent as a HTTP header as part of eachrequest to the server.

Why? Because I was getting confused with lack of fine control over sessions and because the Flask implementationrelied heavily on encryption which seems to be the wrong direction. So we needed a server-side session cookie impl.with fairly fine control.

I intend to replace the existing SqlAlchemy based services with pure psycopg2 implementations, but for now I will becontent not adding another feature to SA

34 Chapter 9. API


Session Cache

The session cache needs to be a fast, distributed lookup system for matching a random ID to a dict of user details.

We shall store the user details in the tabl;e session_cache

Discussion

Caches are hard. They need to be very very fast, and in this case distributable. Distributed caches are very very hardbecause we need to ensure they are synched.

I feel redis makes an excellent cache choice in many circumstances - it is blazingly fast for key-value lookups, it issimple, it is threadsafe (as in threads in the main app do not maintain any pooling or thread issues other than openinga socket or keeping it open) and it has decent synching options.

However the synching is serious concern, and as such using a centralised, fast, database will allow us to move toproduction with a secure solution, without the immediate reliance on cache-invalidation strategies.

Overview

We have one single table, session_cache. This stores a json string (as a string, not 9.3 JSON type) as value in akey value pair. The key is a UUID-formatted string, passed in from the application. It is expected we will never see acollission.

We have three commands:

• set_session()

• get_session()

• delete_session()

With this we can test the whole lifecyle as below

Example Usage

We firstly pass in a badly formed id.:

>>> sid = "Dr. Evil">>> get_session(sid)

Traceback (most recent call last): ...

Rhaptos2Error: Incorrect UUID format for sessionid...

OK, now lets use a properly formatted (but unlikely) UUID

>>> sid = "00000000-0000-0000-0000-000000000001">>> set_session(sid, {"name":"Paul"})True>>> userd = get_session(sid)>>> print userd[0]00000000-0000-0000-0000-000000000001>>> delete_session(userd[0])



To do

• greenlets & conn pooling

• wrap returned recordset in dict.

• pg’s UUID type?

Standalone usage

minimalconfd = {"app": {’pghost’:’127.0.0.1’,’pgusername’:’repo’,’pgpassword’:’CHANGEME’,’pgdbname’:’dbtest’}

}

import sessioncachesessioncache.set_config(minimalconfd)sessioncache.initdb()sessioncache._fakesessionusers()sessioncache.get_session("00000000-0000-0000-0000-000000000000"){u’interests’: None, u’user_id’: u’cnxuser:75e06194-baee-4395-8e1a-566b656f6920’, ...}

>>>

rhaptos2.repo.sessioncache.connection_refresh(conn)Connections should be pooled and returned here.

rhaptos2.repo.sessioncache.delete_session(sessionid)Remve from session_cache an existing but no longer wanted session(id) for whatever reason we want to end asession.

Parameters sessionid – Sessionid from cookie

:returns nothing if success.

rhaptos2.repo.sessioncache.exec_stmt(insql, params)trivial ability to run a dm query outside SQLAlchemy.

Parameters

• insql – A correctly parameterised SQL stmt ready for psycopg driver.

• params – iterable of parameters to be inserted into insql

Return a dbapi recordset (list of tuples)

rhaptos2.repo.sessioncache.get_session(sessionid)

Given a sessionid, if it exists, and is “in date” then return userdict (oppostie of set_session)

Otherwise return None (We do not error out on id not found)

NB this depends heavily on co-ordinating the incoming TZ of the DB and the python app server - I am soleyrunnig the check on the dbase, which avoids that but does make it less portable.

rhaptos2.repo.sessioncache.getconn()returns a connection object based on global confd.

This is, at the moment, not a pooled connection getter.

We do not want the ThreadedPool here, as it is designed for “real” threads, and listens to their states, which willbe ‘awkward’ in moving to greenlets.

36 Chapter 9. API


We want a pool that will relinquish control back using gevent calls

https://bitbucket.org/denis/gevent/src/5f6169fc65c9/examples/psycopg2_pool.pyhttp://initd.org/psycopg/docs/pool.html

Return psycopg2 connection objpsycopg2 connection obj conn obj

Return psycopg2.Error or Err

rhaptos2.repo.sessioncache.initdb()A helper function for creating the session table

rhaptos2.repo.sessioncache.maintenance_batch()A holdng location for ways to clean up the session cache over time. These will need improvement and testing.

rhaptos2.repo.sessioncache.run_query(insql, params)trivial ability to run a query outside SQLAlchemy.

Parameters

• insql – A correctly parameterised SQL stmt ready for psycopg driver.

• params – iterable of parameters to be inserted into insql

Return a dbapi recordset (list of tuples)

run_query(conn, “SELECT * FROM tbl where id = %s;”, (15,))

issues: lots.

•No fetch_iterator.

•connection per query(see above)

•We should at least return a dict per row with fields as keys.

rhaptos2.repo.sessioncache.set_session(sessionid, userd)Given a sessionid (generated according to cnxsessionid spec elsewhere) and a userdict store in ses-sion cache with appropriate timeouts.

Parameters

• sessionid – a UUID, that is to be the new sessionid

• userd – python dict of format cnx-user-dict.

Returns True on successful setting.

Can raise Rhaptos2Errors

TIMESTAMPS. We are comparing the time now, with the expirytime of the cookie in the database This reducesthe portability.

This beats the previous solution of passing in python formatted UTC and then comparing on database.

FIXME: bring comaprison into python for portability across cache stores.

rhaptos2.repo.sessioncache.validate_uuid_format(uuidstr)Given a string, try to ensure it is of type UUID.

>>> validate_uuid_format("75e06194-baee-4395-8e1a-566b656f6920")True>>> validate_uuid_format("FooBar")False


https://bitbucket.org/denis/gevent/src/5f6169fc65c9/examples/psycopg2_pool.py

http://initd.org/psycopg/docs/pool.html


9.2.4 rhaptos2.repo.weblogging

author [email protected] <Paul Brian>

This is initially a simple URL to be listened to:

/logging

The logging endpoint will take either of the two forms of message below apply it either to the local syslog, which weexpect will be configured to centralise over rsyslogd, or it will take a triple, of the below form and convert it into astasd call to be stored on the graphite database.

logging

This endpoint will capture a JSON encoded POST sent to /logging and will process one of three message types.

Firstly, just a block of text expected to be a traceback or other log-ready message. We would assume the client wouldnot insert user data. There is no expectation of capturing session details here. Why would we want the log to be SSLprotected? Might be an idea?

{‘message-type’:’log’, ‘log-message’:’Traceback ...’,

‘metric-label’: null, ‘metric-value’: null, ‘metric-type’: null;, }

The common metric of simply adding one to a global counter is shown here. We are capturing the number of timesanyone types in the word penguin.

{‘message-type’:’metric’, ‘log-message’:null,

‘metric-label’: ‘org.cnx.writes.penguin’, ‘metric-value’: null, ‘metric-type’: ‘incr’, }

Here, a third type of message. We can capture a metric that is a specific value, this would be useful in aggregatereporting. It might be amount of time to perform an action, here its wpm.:

{’message-type’:’metric’,’log-message’:null,

’metric-label’: ’org.cnx.wordsperminute’,’metric-value’: 48,’metric-type’: ’timing’;,}

NB The above message is not yet supported.

Improvements

Run this as a WSGI middleware, so it is simple to import into the chain.

Security considerations

Fundamentally no different from any web service. I expect we shall need to use some form of long running token andkeep the conversastions in SSL to prevent simplistic DDOS attacks.

Simple testing

>>> from weblogging import *>>> confd = {’globals’:... {’syslogaddress’:"/dev/log",

38 Chapter 9. API

mailto:[email protected]


... ’statsd_host’:’log.frozone.mikadosoftware.com’,

... ’statsd_port’:8125,

... }}>>> testmsg = ’’’{"message-type":"log",... "log-message":"This is log msg",... "metric-label": null,... "metric-value": null,... "metric-type": null... }’’’>>> configure_weblogging(confd)>>> logging_router(testmsg)

### FIXME - there is no really decent way to snaffle syslogs in a unit test...

rhaptos2.repo.weblogging.log_endpoint(payload)given a dict, log it to syslog

rhaptos2.repo.weblogging.logging_router(json_formatted_payload)pass in a json message, this will check it, then action the message.

We have several types of incoming message, corresponding to an atc log message, an atc metric message (iegraphite). We want to correctly handle each so this acts as a router/dispatcher

rhaptos2.repo.weblogging.metric_endpoint(payload)given a dict, fire off to statsd

rhaptos2.repo.weblogging.validate_msg_return_dict(json_formatted_payload)

>>>>>> payload_good = ’’’{"message-type":"log",... "log-message":"This is log msg",... "metric-label": null,... "metric-value": null,... "metric-type": null... }’’’>>> x = validate_msg_return_dict(payload_good)>>> x({u’metric-type’: None, u’metric-value’: None, u’metric-label’: None, u’message-type’: u’log’, u’log-message’: u’This is log msg’}, True)

9.2.5 Future Developments

• registration on user service

• API Tokens and user service

• reliance by other services on user service logged in (single Sign on)

9.3 Common functionality

For various reasons the common functions like errors and config are held not in a seperate repo but here.

9.3. Common functionality 39


9.4 rhaptos2.repo.backend

9.5 rhaptos2.repo.configuration

Contains a common configuration parsing class and various utilities for dealing with configuration.

class rhaptos2.repo.configuration.Configuration(settings={}, **sections)A configuration settings object This primarily used to read configuration from file.

classmethod from_file(file, app_name=’app’)Initialize the class from an INI file. The app_name (defaults to DEFAULT_APP_NAME) is used to signifythe main application section in the configuration INI. The application section is put into top-level mapping.All other sections are put in the mapping as a keyed section name and a sub-dictionary containing thesections key value pairs.

>>> ini = ’’’[app]... appkey=appval...... [test]... foo=1...... [test2]... bar=1... ’’’>>> f = "/tmp/foo.ini">>> open(f, "w").write(initxt)>>> C = Configuration.from_file(f)>>> expected = {’test’: {’foo’: ’1’},... ’test2’: {’bar’: ’1’},... "appkey":"appval"}>>> assert C == expected>>> assert C.test == {’foo’: ’1’}>>> assert C.appkey == "appval">>> assert C.test["foo"] == ’1’

9.6 rhaptos2.repo.err

9.7 rhaptos2.repo.log

9.8 run

How to run the repo, with different options.

9.9 rhaptos2.repo.run

Commandline utilities

Contains commandline utilities for initializing the database (initialize_database) and an application for usewith PasteDeploy.

40 Chapter 9. API


rhaptos2.repo.run.initialize_database(argv=None)Initialize the database tables.

rhaptos2.repo.run.paste_app_factory(global_config, **local_config)Makes a WSGI application (in Flask this is app.wsgi_app) and wraps it to serve the static web files.

9.9. rhaptos2.repo.run 41


42 Chapter 9. API

CHAPTER 10

Misc.

Here are misc notes that need to be better incorporated into the body of the docs.

10.1 Glossary

user_detail dict

{user_id user_uri}

1. Concerns over use of <li> in storing data.

We are using textual representations of HTML5 to store a module. This means we store the HTML5 of a moduleas part of a document that represents that doc and its associated metadata.

THis seems to work well.

We are also storing a collection using HTML5 in the body of the documnet - that is the tree structure of acollection is represented in one documnet as a seires of <li> nodes.

Using <li> as nodes is of minor consequence, but there is consequence for storing the whole tree in one doc-ument. Let us take for example a collection of three levels deep - lets choose the article on penguins in theEncycloipaedia Britiannica. THe collection looks like:

Britannica|- P-O|- Penguin

Now if Britannica is a collection (of all the volumes), and stores the wholetree within itself, and the P-O is another collection and stores the wholetree, we have two trees pointing to Penguion - and they need to be kept insynch.

We basically cannot nest collections and store the whole tree within each

43


44 Chapter 10. Misc.

Python Module Index

rrhaptos2.repo.auth, 31rhaptos2.repo.configuration, 40rhaptos2.repo.err, 40rhaptos2.repo.run, 40rhaptos2.repo.sessioncache, 34rhaptos2.repo.views, 29rhaptos2.repo.weblogging, 38

45

release 0.0 · 1.we do not have sqllite working [#]_ as a backend so we must have netowrked...

Documents