| 1 | [[TOC]] |
| 2 | |
| 3 | = New API Implementation Notes = |
| 4 | |
| 5 | This page keeps track of the internals of the Deter API implementation. |
| 6 | |
| 7 | == Configuration Files == |
| 8 | |
| 9 | Both the test utilities and the service itself have configuration files. |
| 10 | |
| 11 | === Service Configurations === |
| 12 | |
| 13 | Service configuration file a are java properties files stored in /usr/local/etc/deter/. The main configuration file is |
| 14 | |
| 15 | /usr/local/etc/deter/service.properties which consists of (passwords and usernamed expunged): |
| 16 | |
| 17 | {{{ |
| 18 | #DETER utilities properties |
| 19 | #Thu Jul 18 16:27:29 PDT 2013 |
| 20 | emulabDbUrl=jdbc\:mysql\://localhost/tbdb?user\=auser&password\=apassword |
| 21 | keystorefilename=/usr/local/apache-tomcat-7.0/conf/tomcat.keystore |
| 22 | logconfig=/usr/local/etc/deter/log4j.properties |
| 23 | deterDbUrl=jdbc\:mysql\://localhost/deter?user\=auser&password\=apassword |
| 24 | keystorepw=apassword |
| 25 | }}} |
| 26 | |
| 27 | Contents are: |
| 28 | |
| 29 | * the URL to read and write the emulab DB (emulabDbUrl) |
| 30 | * the URL to read and write the deter DB (deterDbUrl) |
| 31 | * the keystore containing the id used to sign new credentials (keystore) |
| 32 | * the peystore passowrd (keystorepw) |
| 33 | * the log4j properties file used to configure the service logging (logconfig) |
| 34 | |
| 35 | The loggingconfiguration syntax is defined by [http://logging.apache.org/log4j/1.2/manual.html log4j]. Current contents are: |
| 36 | |
| 37 | {{{ |
| 38 | # logging for DeterServices |
| 39 | # |
| 40 | |
| 41 | # For debugging this configuration set this variable true and look at tomcat's |
| 42 | # stderr. |
| 43 | log4j.debug=true |
| 44 | |
| 45 | # Rolling log appender to put out into /var/log/deter |
| 46 | log4j.appender.A1=org.apache.log4j.RollingFileAppender |
| 47 | log4j.appender.A1.MaxFileSize=10MB |
| 48 | log4j.appender.A1.MaxBackupIndex=3 |
| 49 | log4j.appender.A1.layout=org.apache.log4j.PatternLayout |
| 50 | log4j.appender.A1.layout.ConversionPattern=%d{MM/dd HH:mm:ss} %-5p %c{1}: %m%n |
| 51 | log4j.appender.A1.file=/var/log/deter/service.log |
| 52 | |
| 53 | # The net.deterlab logger will inherit the root level and use the |
| 54 | # /var/log/deter appender above. It will not log to the root. |
| 55 | log4j.logger.net.deterlab=DEBUG, A1 |
| 56 | log4j.additivity.net.deterlab=false |
| 57 | |
| 58 | }}} |
| 59 | |
| 60 | That defines a service log in /var/log/deter/services.log logging at DEBUG and higher. It is automatically rotated at 10 MB with 3 copies kept around. We use the log4j rotator rather than newsyslog to avoid disruption. |
| 61 | |
| 62 | === Utility Configurations === |
| 63 | |
| 64 | Utilities read from a configuration in the users {{{$HOME/.deterutils.properties}}}. This is a java properties file. A simple setup looks like (passwords and users purged): |
| 65 | |
| 66 | {{{ |
| 67 | #DETER utilities properties |
| 68 | #Wed Aug 14 18:09:13 PDT 2013 |
| 69 | useridpw=apassword |
| 70 | serviceurl=https\://localhost\:52323/axis2/services/ |
| 71 | trustpw=apassword |
| 72 | useridfilename=/usr/home/faber/myID |
| 73 | trustfilename=/usr/home/faber/keystore |
| 74 | }}} |
| 75 | |
| 76 | This keeps track of |
| 77 | |
| 78 | * Trusted certificates - the DETER service id (trustfilename & trustpw) |
| 79 | * User identity certificate - useridfilename, useridpw) |
| 80 | * The service base URL - ( serviceurl) |
| 81 | |
| 82 | Both files can be modified by the SetProperty utility. |
| 83 | |
| 84 | |
| 85 | == Database tables == |
| 86 | |
| 87 | === Users === |
| 88 | |
| 89 | Users are kept in a table of this format called users: |
| 90 | |
| 91 | |
| 92 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 93 | ||idx || int(11) || NO || PRI || NULL || auto_increment || |
| 94 | ||uid || varchar(20) || YES || || NULL || || |
| 95 | ||password || varchar(255) || YES || || NULL || || |
| 96 | ||hashtype || varchar(32) || YES || || NULL || || |
| 97 | ||passwordexpires || datetime || YES || || NULL || || |
| 98 | |
| 99 | Most of that is straightforward. The password is a hash with type defined by hashtype. Currently all are crypt format hashes. |
| 100 | |
| 101 | Passowrds are checked using a challenge system and password resets also populate the same table. When created a challenge is put in the userchallenge table: |
| 102 | |
| 103 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 104 | || uidx || int(11) || YES || || NULL || || |
| 105 | || data || blob || YES || || NULL || || |
| 106 | || validity || datetime || YES || || NULL || || |
| 107 | || challengeid || bigint(20) || YES || UNI || NULL || || |
| 108 | || type || varchar(20) || YES || || NULL || || |
| 109 | |
| 110 | A password challenge - checking a login - has type "clear." The data is the challenge issued with enough info to check the response. Currently, only a clear challenge is used, meaning that the data contains the crypt hash of the password to check. The validity field is checked and old challenges discarded whenever a challeng is accessed. uidx is the index (idx) of the user for whom the challenge was issued. As with all DB fields the links are through integer indices. |
| 111 | |
| 112 | When used to reset a password the challenge is issued with type "PasswordReset". The presence of such a challenge is all that is checked. A user who calls changePasswordChallenge with a the ID of a valid "PasswordReset" challenge can change the password of the user pointed to by the uidx field. |
| 113 | |
| 114 | The body of a notification is stored in the notification table: |
| 115 | |
| 116 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 117 | || idx || int(11) || NO || PRI || NULL || auto_increment || |
| 118 | || created || timestamp || NO || || CURRENT_TIMESTAMP || on update CURRENT_TIMESTAMP || |
| 119 | || body || text || YES || || NULL || || |
| 120 | |
| 121 | When delivered to a user a row is added to the usernotification table for each user: |
| 122 | |
| 123 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 124 | || nidx || int(11) || YES || MUL || NULL || || |
| 125 | || uidx || int(11) || YES || MUL || NULL || || |
| 126 | || flags || int(11) || YES || || NULL || || |
| 127 | |
| 128 | |
| 129 | nidx and uidx are both constrained to be foreign key indices unto the notification and users tables respectively. This means that all notifications to a user are removed before the user is removed. Each entry in the usernotification table is a status for that message to a user (urgent/read). |
| 130 | |
| 131 | === Profiles === |
| 132 | |
| 133 | Most objects in the Deter API have a profile attached to them and all of these tables are the same with minor tweaks. Here is the schema for user profile entries (the userattribute table): |
| 134 | |
| 135 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 136 | || idx || int(11) || NO || PRI || NULL || auto_increment || |
| 137 | || name || varchar(20) || YES || || NULL || || |
| 138 | || datatype || enum('STRING','INT','FLOAT','OPAQUE') || YES || || NULL || || |
| 139 | || optional || tinyint(4) || YES || || NULL || || |
| 140 | || access || enum('READ_WRITE','READ_ONLY','WRITE_ONLY','NO_ACCESS') || YES || || NULL || || |
| 141 | || description || text || YES || || NULL || || |
| 142 | || format || varchar(256) || YES || || NULL || || |
| 143 | || formatdescription || text || YES || || NULL || || |
| 144 | || sequence || int(11) || NO || || 0 || || |
| 145 | || length || int(11) || NO || || 0 || || |
| 146 | |
| 147 | This is a description of the attribute. Note that attribute descriptions have indices. |
| 148 | |
| 149 | To assign a value in a given user's profile, an entry is added to the userattributevalue table: |
| 150 | |
| 151 | ||= Field =||= Type =||= Null =||= Key =||= Default =||= Extra =|| |
| 152 | || uidx || int(11) || YES || MUL || NULL || || |
| 153 | || aidx || int(11) || YES || MUL || NULL || || |
| 154 | || value || text || YES || || NULL || || |