Changes between Initial Version and Version 1 of NewImplNotes


Ignore:
Timestamp:
Aug 20, 2013 12:12:32 PM (11 years ago)
Author:
faber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • NewImplNotes

    v1 v1  
     1[[TOC]]
     2
     3= New API Implementation Notes =
     4
     5This page keeps track of the internals of the Deter API implementation.
     6
     7== Configuration Files ==
     8
     9Both the test utilities and the service itself have configuration files.
     10
     11=== Service Configurations ===
     12
     13Service configuration file a are java properties files stored in /usr/local/etc/deter/.  The main configuration file is
     14
     15/usr/local/etc/deter/service.properties which consists of (passwords and usernamed expunged):
     16
     17{{{
     18#DETER utilities properties
     19#Thu Jul 18 16:27:29 PDT 2013
     20emulabDbUrl=jdbc\:mysql\://localhost/tbdb?user\=auser&password\=apassword
     21keystorefilename=/usr/local/apache-tomcat-7.0/conf/tomcat.keystore
     22logconfig=/usr/local/etc/deter/log4j.properties
     23deterDbUrl=jdbc\:mysql\://localhost/deter?user\=auser&password\=apassword
     24keystorepw=apassword
     25}}}
     26
     27Contents are:
     28
     29 * the URL to read and write the emulab DB (emulabDbUrl)
     30 * the URL to read and write the deter DB (deterDbUrl)
     31 * the keystore containing the id used to sign new credentials (keystore)
     32 * the peystore passowrd (keystorepw)
     33 * the log4j properties file used to configure the service logging (logconfig)
     34
     35The loggingconfiguration syntax is defined by [http://logging.apache.org/log4j/1.2/manual.html log4j].  Current contents are:
     36
     37{{{
     38# logging for DeterServices
     39#
     40
     41# For debugging this configuration set this variable true and look at tomcat's
     42# stderr.
     43log4j.debug=true
     44
     45# Rolling log appender to put out into /var/log/deter
     46log4j.appender.A1=org.apache.log4j.RollingFileAppender
     47log4j.appender.A1.MaxFileSize=10MB
     48log4j.appender.A1.MaxBackupIndex=3
     49log4j.appender.A1.layout=org.apache.log4j.PatternLayout
     50log4j.appender.A1.layout.ConversionPattern=%d{MM/dd HH:mm:ss} %-5p %c{1}: %m%n
     51log4j.appender.A1.file=/var/log/deter/service.log
     52
     53# The net.deterlab logger will inherit the root level and use the
     54# /var/log/deter appender above.  It will not log to the root.
     55log4j.logger.net.deterlab=DEBUG, A1
     56log4j.additivity.net.deterlab=false
     57
     58}}}
     59
     60That defines a service log in /var/log/deter/services.log logging at DEBUG and higher.  It is automatically rotated at 10 MB with 3 copies kept around.  We use the log4j rotator rather than newsyslog to avoid disruption.
     61
     62=== Utility Configurations ===
     63
     64Utilities read from a configuration in the users {{{$HOME/.deterutils.properties}}}.  This is a java properties file. A simple setup looks like (passwords and users purged):
     65
     66{{{
     67#DETER utilities properties
     68#Wed Aug 14 18:09:13 PDT 2013
     69useridpw=apassword
     70serviceurl=https\://localhost\:52323/axis2/services/
     71trustpw=apassword
     72useridfilename=/usr/home/faber/myID
     73trustfilename=/usr/home/faber/keystore
     74}}}
     75
     76This keeps track of
     77
     78 * Trusted certificates - the DETER service id (trustfilename & trustpw)
     79 * User identity certificate - useridfilename, useridpw)
     80 * The service base URL - ( serviceurl)
     81
     82Both files can be modified by the SetProperty utility.
     83
     84
     85== Database tables ==
     86
     87=== Users ===
     88
     89Users are kept in a table of this format called users:
     90
     91
     92||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     93||idx || int(11) || NO   || PRI || NULL  || auto_increment ||
     94||uid || varchar(20) || YES  ||    || NULL || ||
     95||password || varchar(255) || YES  ||  || NULL || ||
     96||hashtype || varchar(32)  || YES  ||   || NULL || ||
     97||passwordexpires || datetime   || YES ||     || NULL ||  ||
     98
     99Most of that is straightforward.  The password is a hash with type defined by hashtype.  Currently all are crypt format hashes.
     100
     101Passowrds are checked using a challenge system and password resets also populate the same table.  When created a challenge is put in the userchallenge table:
     102
     103||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     104|| uidx || int(11) || YES || || NULL || ||
     105|| data || blob || YES || || NULL || ||
     106|| validity || datetime || YES || || NULL || ||
     107|| challengeid || bigint(20) || YES || UNI || NULL || ||
     108|| type || varchar(20) || YES || || NULL || ||
     109
     110A password challenge - checking a login - has type "clear." The data is the challenge issued with enough info to check the response.  Currently, only a clear challenge is used, meaning that the data contains the crypt hash of the password to check.  The validity field is checked and old challenges discarded whenever a challeng is accessed.  uidx is the index (idx) of the user for whom the challenge was issued.  As with all DB fields the links are through integer indices.
     111
     112When used to reset a password the challenge is issued with type "PasswordReset".  The presence of such a challenge is all that is checked.  A user who calls changePasswordChallenge with a the ID of a valid "PasswordReset" challenge can change the password of the user pointed to by the uidx field.
     113
     114The body of a notification is stored in the notification table:
     115
     116||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     117|| idx || int(11) || NO || PRI || NULL || auto_increment ||
     118|| created || timestamp || NO || || CURRENT_TIMESTAMP || on update CURRENT_TIMESTAMP ||
     119|| body || text || YES || || NULL || ||
     120
     121When delivered to a user a row is added to the usernotification table for each user:
     122
     123||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     124|| nidx || int(11) || YES || MUL || NULL || ||
     125|| uidx || int(11) || YES || MUL || NULL || ||
     126|| flags || int(11) || YES || || NULL || ||
     127
     128
     129nidx and uidx are both constrained to be foreign key indices unto the notification and users tables respectively.  This means that all notifications to a user are removed before the user is removed.  Each entry in the usernotification table is a status for that message to a user (urgent/read).
     130
     131=== Profiles ===
     132
     133Most objects in the Deter API have a profile attached to them and all of these tables are the same with minor tweaks.  Here is the schema for user profile entries (the userattribute table):
     134
     135||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     136|| idx || int(11) || NO || PRI || NULL || auto_increment ||
     137|| name || varchar(20) || YES || || NULL || ||                               
     138|| datatype || enum('STRING','INT','FLOAT','OPAQUE') || YES || || NULL || ||
     139|| optional || tinyint(4) || YES || || NULL || ||                             
     140|| access || enum('READ_WRITE','READ_ONLY','WRITE_ONLY','NO_ACCESS') || YES || || NULL || ||
     141|| description || text || YES || || NULL || ||
     142|| format || varchar(256) || YES || || NULL || ||                             
     143|| formatdescription || text || YES || || NULL || ||
     144|| sequence || int(11) || NO || || 0 || ||                                   
     145|| length || int(11) || NO || || 0 || ||
     146
     147This is a description of the attribute.  Note that attribute descriptions have indices.
     148
     149To assign a value in a given user's profile, an entry is added to the userattributevalue table:
     150
     151||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =||
     152|| uidx || int(11) || YES || MUL || NULL || ||
     153|| aidx || int(11) || YES || MUL || NULL || ||
     154|| value || text || YES || || NULL || ||