Stephen Swann

Tuesday, 13 July 2010
In my "Twitter and ITDI" article I showed how we can use ITDI to send a tweet and I stated that getting ITDI to read tweets might be a good next step in the process of fully Twitter-enabling our ITDI Assembly Lines. This article will hopefully shed some light on how we could go about doing just that.

The first thing we need to do is create ourselves an ITDI Assembly Line and drop in a FOR-EACH Connector Loop of type HTTP Client. I configured my HTTP Client connector to perform a GET on the following URL:

http://api.twitter.com/1/statuses/user_timeline.xml?screen_name=stephenjswann&count=5

I slected the Simple XML Parser as a parser for this connector and assigned the Root Tag and Entry Tag as "statuses" and "status" respectively, as such:

I performed a connection on my Input Map tab, clicked on next and dragged the created_at and text schema attributes to my map.

Fantastic! I now have a connector which will loop around the tweets it receives as a result of performing a GET request on the user_timeline.xml resource on Twitter. So what next? Well, within the loop, I would probably want to drop in an IF branch with the following script in place:


This script will query the contents of each tweet looking for the hashtag #tditest. Any tweet containing this hashtag will invoke the contents of our IF branch. Of course, you could put whatever you want into the IF branch but for the purposes of my test, I dropped in the sendTweet function used in yesterday's article to post a response back to Twitter stating that a tweet with the #tditest hashtag had been found. I'm quite sure there are many more practical actions which could be performed at this stage!

Now the intelligencia reading this will be wondering what use this loop connector is in its present form. After all, once we have read through the tweets that we have requested, the Assembly Line will stop. And if we run it again, it will just process the same tweets again, right?

Right. Unless we expand our solution with a couple of minor tweaks. We can enable the delta processing in our HTTP Client Connector which ensures that we will only ever process NEW tweets. Any unique attribute of the tweet should suffice (and there are a few) but I settled on created_at, for no particular reason:
Now I can run my Assembly Line over and over again safe in the knowledge that it won't trigger my IF branch at all unless there is a new Tweet that it hasn't ever seen before. The final step for me was to wrap this Assembly Line in a timer controlled Assembly Line. I created a readTweetsController Assembly Line with a timer function as an iterator and all time attributes set to * (which means the AL will fire every minute). In my data flow, I placed an Assembly Line Function component to call the AL I built above.

The result? A mechanism which polls Twitter every minute looking for new tweets with a particular hashtag and "does stuff" if a matching tweet is found.

So what are the real world applications for such a mechanism? Theoretically, I could:

• post a tweet "TDI: getStatistics" and my TDI AL could return "TDI RESPONSE: 20,000 records processed"
  post a tweet "TDI: startSync" and my TDI AL would kick off a synchronisation process and return "TDI RESPONSE: synchronisation started" followed some time later with "TDI RESPONSE: synchronisation complete"
• post a tweet "TDI: switchOnLights" and my TDI AL could switch on the lights in my house
  post a tweet "TDI: exec "su - shutdown now" and my TDI AL would ignore me as a fool

Of course, there may be better ways of doing any or all of these tasks but using Twitter for non-critical communcations with your ITDI Assembly Lines should be relatively straightforward. After all, we aren't always within easy reach of our laptops, but we rarely go anywhere without our phones and remote management via phone and Twitter could be useful.

Hopefully this article will give you some food for thought.

Monday, 12 July 2010
It is becoming clear that Twitter isn't just a platform for telling the world what you are having for your dinner. Twitter is used as a marketing tool, a news information service and a tool for comedians to test new jokes. It's robustness and message persistence make it an excellent candidate for creating a Message Queue service for those who haven't the inclination for deploying a "proper" MQ service and it can also be used as an alerting mechanism.

For example, I would be quite happy to receive a tweet stating that something had happened to my enterprise application during the night with relevant information like an error code. After all, it is a free way to send the equivalent of a SMS.

Twitter offers a great API and a simple single command can update your Twitter status as such:

curl -u user:password -d "status=This is my first tweet via the Command Line"

But what if curl isn't available on your system? What if you have a suite of ITDI Assembly Lines which you would like to interface with Twitter? Well, nothing could be simpler (until Twitter disable their Basic Authentication mechanism).

An HTTP Client Connector should be created in LookUp mode within your Assembly Line and the connection details updated as such:

The Link Criteria should be updated to include a status parameter:


And the returning http.body attribute can be converted into a string for further analysis such as checking that the tweet was sent successfully:


Our sendTweet connector would be a great addition to our arsenal of alerting mechanisms which should already include sendMail, sendSNMPAlert and updateLog.

Next Steps

Building a Twitter Assembly Line which can perform the OAuth authentication would be a natural next step as would a connector which can read a "twitter stream" (as the equivalent of an MQ Consumer).

NOTE: The above screenshots were taken from a TDI v7.0 instance running on Windows 2003 but the concepts can be used in older versions of TDI with little (or no) modification.

Monday, 5 July 2010
Most of the time, getting IBM Tivoli Identity Manager to provision accounts successfully in target systems is a breeze. And you could be forgiven for thinking that provisioning to IBM Tivoli Access Manager (another stalwart of the IBM Tivoli portfolio) would be the simplest of the lot.

You would, of course, be wrong. At least, if you want to make use of GSO credentials, you would be wrong. Maybe. Possibly.

You see, the provisioning of GSO credentials just doesn't seem to be that easy!

This article assumes the following:

• the reader has a basic understanding of IBM Tivoli Identity Manager
• the reader has a basic understanding of IBM Tivoli Access Manager
• the reader understands GSO credentials within TAM
• the version of ITIM being used is v5.0 (or higher)
• the version of TAM being used is v6.0 (or higher)
• the TAM Combo Adapter is being used (v5.0.9)

For an explanation of the problem, let us assume that a resource has been created within the TAM domain called backed. In pdadmin, we can create this resource as such:


In ITIM, we have a provisioning policy created with all the standard default TAM attributes being populated (such as cn, sn, our TAM groups, etc.). We check the box for Single Sign On Capability which leaves us with the Resource Credentials to populate.

In our environment, we have password synchronisation enabled across all systems and the GSO credentials will be no different. In other words, should a user change their password in ITIM, their TAM password and their GSO resource credential passwords will also be updated.

The "Constant Value" option for the Resource Credentials on our Provisioning Policy form is of no help for a User ID and Password that will be different for each user:


Which leaves us with the option of scripting the credentials. The documentation provided with the TAM Combo Adapter at least tells us that the TAM Resource Credentials must be provided in the following format:


In our case, that means that our javascript will look something like this:


We have a problem. We've always known that we can use parameters.eruid[0] for our User ID but what do we use for our password? A little bit of crawling around the documentation tells us that when password synchronisation is enabled, the ersynchpassword can be used to retrieve the password. So our {something} becomes ersynchpassword, right?

Wrong!

In fact, when we do this and provision an account (with a password of auiTNG85) our secAuthData attribute in LDAP contains the following:


How do I know that this is wrong? I know because I manually created the Resource Credential via the pdadmin command prompt and know that the result I'm looking for is:


Indeed, if I hardcode my Provisioning Policy to set {something} to my password (auiTNG85), I get the same incorrect result. Odd? Not really. If you dig further into the documentation you will see that there is some funny business going on with the TAM Combo Adapter and the password in the Resource Credential attribute on the Provisioning Policy MUST be prefixed with {clear}. In our hardcoded entitlement, we would therefore have:


Now, when we create our account, the secAuthnData is set correctly! Of course, I'm hardcoding the password still so I need to do something about that! We need to re-introduce our ersynchpassword attribute but ensure we are prefixing is with {clear}. And the reason we do that is because ITIM is just too clever by half and has already Base64 Decoded the attribute on our behalf!


The resulting secAuthnData is now set to:


Perfection. I'm sure the documentation contains all the information required to get this result but it isn't instantly clear that this is how you should be going about the GSO Credential Password Synchronisation problem.

Sunday, 4 July 2010
IBM Tivoli Identity Manager can be a beast at times. On the face of it, having a tool that can manage your account objects in various data repositories/systems doesn't sound like it ought to be complicated. However, the reality can be quite tricky. Person objects, account objects, role objects, organisational hierarchy objects, service objects, provisioning policies, identity policies, adoption policies, password policies, entitlements, accesses... that's a lot of data and the relationships these data objects have with each other can get confusing for some.

Person objects own account objects which are provisioned by virtue of an access request or a provisioning policy which contains entitlements granted by role membership for specific services or service types and the accounts' User ID is governed by an Identity Policy, etc.

There are some excellent technical documents available on the IBM website which attempt to explain these objects but I've rarely found a visual description of the objects which works - thus my attempt using Visio:

Now, it should be pointed out that this visual representation is incomplete. How could I possibly have shown ALL the relationship lines without them criss-crossing in a way which would make the diagram "unviewable". For example, almost every object gets "placed" in a Business Unit yet I've only shown a person object belonging to a business unit! However, I hope it helps explain the basic relationships.

If you want the Visio 2007 version of this diagram, you can get it from http://www.stephen-swann.co.uk/downloads/itim-object-relationships.vsd. Enjoy.

NOTE: This diagram refers to IBM Tivoli Identity Manager v5.1

Wednesday, 16 June 2010
I wanted to build a new ITIM environment recently and figured it was about time I started to put together a proper set of Jython scripts to help me automate the process. Below, I've detailed my thinking behind a Jython script which will be capable of taking a delimited file defining an Organisational Structure, and load it into ITIM using ITIM APIs and the Jython scripting framework.

My virtual environment for this purpose was:

• Windows 2003 R2 Standard Edition Service Pack 2
• IBM WebSphere Application Server v7.0.0.9
• IBM Tivoli Identity Manager v5.1.0.0
• APISCRIPT v5.0.0.4 (available from OPAL)

The Setup
The apiscript tool (v5.0.0.4) should be downloaded from the OPAL site and deployed as per the instructions. For example,  the following files need to be configured to match the ITIM environment being managed:

• etc/host/{hostname}.properties
• bin/env_master.bat

But because I'm using an ITIM v5.1 system, I need to also make a "tweak" to the apiscript.bat (or apiscript.ksh) file as the structure of the extensions directory has been updated in this release. I need to include a 5.1 directory between extensions and examples as such:

set APISCRIPT_LOGIN_CONFIG=%APISCRIPT_ITIM_HOME%\extensions\5.1\examples\apps\bin\jaas_login_was.conf

Input File
The following input file is to be used to define the organisational structure:

ParentOrgUnit|OrgUnit|

|internal|

|external|

internal|unit1|

internal|unit2|

external|unit3|

unit2|unit4|

unit2|unit5|

unit10|unit7|

unit4|unit6|

external|unit4|

external|unit8|

external|unit8|

unit8|unit9|

unit9|unit8|

Each line was terminated with a | character because early in the testing process I noticed that carriage return/line feed characters were making their way into the ITIM environment.

The Code

The code required to process such an input file should be broken down into a number of sections:


Section 1: Process The Command Line Arguments
In order to ensure that the script can process a variety of input files, the file to be processed should be passed as a command line argument. An additional argument is being processed here to enable verbose logging to take place:

try:
   opts, args = getopt.getopt(sys.argv[1:], "qi:", ["inputfile="])
except getopt.GetoptError, err:
   print str(err)
   usage()
   sys.exit(2)

inputfile = ""
quietmode = "false"

for opt, arg in opts:
   if opt == "-q":
      print "Quiet mode enabled"
      quietmode = "true"
   elif opt == "-i":
      inputfile = arg

Section 2: Read the Input File
The processing of the input file should be wrapped in a while loop and each line processed should be split into its constituent parts using the split method on the line object:

infile = open(inputfile,"r")
while infile:
   line = infile.readline()
   if not line: break
   items=line.split("|")
   parentorg=items[0]
   org=items[1]

At this point, we now have an Organisational Unit Name and a Parent Organisational Unit Name ready for placement in the ITIM data model.

Section 3: Process Each Entry
For each record retrieved above, we need to determine that the Parent Organisational Unit Name actually exists in the data model. If it does not, we cannot process the entry. If it does exist, we need to check that the child OU doesn't already exist. If it also exists, then there is no point in continuing processing of this entry, otherwise we need to get an OrganizationalUnitMO object for the Parent OU to enable us to create an OU using the child Organisational Unit Name.

if parentorg == 'ParentOrgUnit':
   # Do Nothing - it's the header
   logit('Processing Header')
else:
   logit('**********************************************')
   logit('Processing ' + org)

   if org_exists(parentorg) != 'False':
      if org_exists(org) != 'False':
         logit('Child OU already exists:' + org)
      else:
         # Create OU
         logit('Child OU does not exist:' + org)
         parent_org_mo = get_org_mo(parentorg)
         orgchart.do_create_ou_from_args(org, parent_org_mo)
      # end if loop
   else:
      logit('Parent Org Unit does not exist:' + parentorg)
   # end if loop
# end if

Section 4: Check That An OU Exists
To check that an OU exists, we need to perform a search using a SearchMO object for an ORGUNIT object using a filter based on the OU name provided in the data file. If an object is found, we should return TRUE, else return FALSE:

def org_exists(orgunit):
   foundit = 'False'
   if orgunit == "":
      foundit = 'true'
   else:
      def_org_cont_mo = orgchart.get_default_org_mo()
      myPlatform = apiscript.util.get_default_platform_ctx_and_subject()
      search_mo = SearchMO(myPlatform)
      search_mo.setCategory(ObjectProfileCategoryConstant.ORGUNIT)
      myFilter = "(ou=" + orgunit + ")"
      search_mo.setFilter(myFilter)
      search_mo.setScope(SearchParameters.SUBTREE_SCOPE)
      search_mo.setContext(def_org_cont_mo)
      results_mo = search_mo.execute()
      for result in results_mo.getResults():
         if orgunit == result.name:
            foundit = 'True'
   return foundit
# end org_exists

Section 5: Get the MO Of An OU
An OrganizationalUnitMO object can be generated after searching for an OU by performing two additional functions:

• getDistinguishedName on the search result
• create_org_container_mo using the DN returned from the above method

def get_org_mo(orgunit):
   if orgunit == "":
      org_mo = orgchart.get_default_org_mo()
   else:
      def_org_cont_mo = orgchart.get_default_org_mo()
      search_mo = SearchMO( *apiscript.util.get_default_platform_ctx_and_subject())
      search_mo.setCategory(ObjectProfileCategoryConstant.ORGUNIT)
      myFilter = "(ou=" + orgunit + ")"
      search_mo.setFilter(myFilter)
      search_mo.setScope(SearchParameters.SUBTREE_SCOPE)
      search_mo.setContext(def_org_cont_mo)
      results_mo = search_mo.execute()
      for result in results_mo.getResults():
         if orgunit == result.name:
            mydn = result.getDistinguishedName()
            org_mo = orgchart.create_org_container_mo(mydn)
   return org_mo
# End get_org_mo

The Result
The output from the script is:

C:\work\apiscript-5.0.0.4\apiscript>c:\work\apiscript-5.0.0.4\apiscript\bin\apiscript.bat -f c:\work\apiscript-5.0.0.4\apiscript\py\orgs.py -z -i c:\\work\\apiscript-5.0.0.4\\apiscript\\data\\orgs.dat
Using master environment: "C:\work\apiscript-5.0.0.4\apiscript\bin\env_master.bat"
Using custom BIN_HOSTNAME: "stephen-w0ckd5b"
Using custom ETC_HOSTNAME: "stephen-w0ckd5b"
Using host properties: "C:\work\apiscript-5.0.0.4\apiscript\etc\host\stephen-w0ckd5b.properties"
Using APISCRIPT_WAS_HOME: "C:\Program Files\IBM\WebSphere\AppServer"
Using APISCRIPT_ITIM_HOME: "C:\Program Files\IBM\itim"
WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
Welcome to IBM Tivoli Identity Manager API Scripting Tool (apiscript) version: 5.0.0.4
Setting system property: java.security.auth.login.config
Setting com.ibm.CORBA properties: loginSource, loginUserid, loginPassword
WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[-z, -i, c:\\work\\apiscript-5.0.0.4\\apiscript\\data\\orgs.dat]"
Logging configuration file is not found. All the logging information will be sent to the console.
**********************************************
Input file selected is c:\work\apiscript-5.0.0.4\apiscript\data\orgs.dat
Processing Header
**********************************************
Processing internal
Child OU does not exist:internal
**********************************************
Processing external
Child OU does not exist:external
**********************************************
Processing unit1
Child OU does not exist:unit1
**********************************************
Processing unit2
Child OU does not exist:unit2
**********************************************
Processing unit3
Child OU does not exist:unit3
**********************************************
Processing unit4
Child OU does not exist:unit4
**********************************************
Processing unit5
Child OU does not exist:unit5
**********************************************
Processing unit7
Parent Org Unit does not exist:unit10
**********************************************
Processing unit6
Child OU does not exist:unit6
**********************************************
Processing unit4
Child OU already exists:unit4
**********************************************
Processing unit8
Child OU does not exist:unit8
**********************************************
Processing unit8
Child OU already exists:unit8
**********************************************
Processing unit9
Child OU does not exist:unit9
**********************************************
Processing unit8
Child OU already exists:unit8

C:\work\apiscript-5.0.0.4\apiscript>

Visually, this gets represented in ITIM as:


In conclusion, I'm not a Jython/Python sripting guru. Indeed, this was my first foray into Python. It is, however, a relatively straightforward language to learn and can be a very powerful tool in your ITIM Administrative toolset.

The full script can be downloaded at downloads/loadous.zip.