MS 2.0

Web and Microsoft has come a long way in the past decade. Remember this?

This site is best viewed on IE in 1024x768 resolution

Red + Blue

I moved one of my toy cloud app from Redhat to Bluemix and then this hapenned...

What a big shift in Microsoft's attitude towards open source

Open source contributions made by employees of different organizations

Courtesy: https://octoverse.github.com/projects

One of those things that hasn't changed in the past 15 years

Cutting the corners

No day passes by without a discussion of the traffic woes during the lunch. Everyone narrates their ordeal during the morning commute and we all end up agreeing that this is not going to change. If at all, it's going to become worse. The location is Chennai but similar discussions are common place around India. No amount of blaming or outrage is going to change the situation, instead i just want to document some common mistakes on the road I encounter regularly.

Being a non-native English speaker, I encountered the term "cutting the corners" in a different context and only later realized that the term could have originated by the behaviour on road.

Cutting corners #1

Cutting corners #2

setuptools - dealing with compiling custom modules on windows

pip install has hugely simplified how python modules are installed and managed but sometimes it simply can't compile custom extensions despite having all the prerequisites. This simply boils down to pip not being able to automatically determine the required visual studio environment. The simplest fix I found and that I keep forgetting is to get to visual studio command prompt and set a couple of environment variables to prevent distutils from auto discovering the visual studio environment.

%path_to%\vcvarsall.bat x64

SET DISTUTILS_USE_SDK=1
SET MSSdk=1

Pausing...

This thought came up one day when I was in my second job. My first job was all about exciting stuff such as device drivers, firmware for smart card terminals, cryptography and a whole lot of cutting edge technologies, the fancy and glamorous stuff that would be a dream for many aspiring programmers. But I cannot say the same thing about my second job. To my surprise I didn't even realize it till very long into my job. Motorola 68k, serial communication, matrix keypad and text display can’t be called cutting edge even 15 years back, that’s when this thought popped up - What was I doing all these 6 years? Was I in a long hibernation, isolated and lost in the career path? I sure was not idle all these years, rather I was very busy fixing things, so it definitely should have some value addition and learnings that I didn’t realize. That’s when I set out to reflect on my experience working on legacy software and hardware platforms.

Self driving cars - the Indian dilemma

I read an article about self driving cars which got me thinking about the prospects of such a car in an Indian context. Aside from the general problems that autonomous cars have to tackle, I could instantly think of  a few additional and unique problems of its own in an Indian context.

The direction dilemma: At times when I end up in a new intersection, I have a few problems waiting to be solved before I can proceed towards the destination.

• Missing signal posts
• Misaligned signal posts
• Signal post not visible from the stop line because it's sometimes right over the head and even behind it.

The stop dilemma: On one occasion, the car in front stopped and I stopped behind it as well but later realized that the car has "pulled off" for a break without any indication.

The indicator dilemma: How many times I have driven behind a vehicle with the indicator on assuming that it will take a turn but later realized that it wouldn't because the driver had forgot to switch it off after the last turn.

The pass dilemma: I have wondered for a long time why the switch to blink the headlight is marked as "pass" on my motorcyle. In India this is used for the exactly opposite purpose, i.e. to THREATEN the oncomers to STOP when one wants to take a turn.

The indicator dilemma: Driving on highways, it took me some time to realize that the turn indicator is used by a vehicle in the front to signal it's intention to allow the vehicle behind it to overtake.

The honking dilemma: I am riding and not doing anything wrong and some one honks from behind - what should I do.

These are just a few scenarios that came to my mind instantly. SURPRISES are the norm here, even an experienced driver has to think twice before making a decision, leave alone the self driving cars. Last night I was taken by surprise when I saw the display board on a minibus flashing green LEDs and for a second thought that it was the traffic signal. U-turns, I don't even want to talk about it. Many assumptions that is a starting point for the self driving cars will be rendered untrue, we need to raise to the ground fist before building on top of it. The irony is that, these cars will one day teach us how to behave on the road!

The first Ubuntu Phone

Eagerly awaiting more details to emerge on the first Ubuntu phone - BQ Aquaris E4.5. Especially on the software and how Canonical has managed to build the GUI stack and multimedia playback. Given that the Linux support is not something to cheer about when it comes to graphics drivers and video decoding, the fact that there's now a real consumer device is something interesting and awaiting some real reviews on the device. There has been quite a bit of discussions on the Mir Display Server and libhybris - Waiting to see how these have shaped up for the phone.

A tribute to my PC

Back in 2004, I bought my first Personal Computer. It was assembled by a local seller and had a Gigabyte motherboard with a VIA K8M800 chipset and a Socket 754 for AMD Athlon 64 bit processor. I was very excited and was eagerly awaiting the delivery and this was around the Pooja holidays. One fine night it arrived, the PC and it's accompanies - a Samsung CRT monitor, a Samsung PS/2 Keyboard with multimedia keys and a dedicated sleep/shutdown button, a Logictech PS/2 Optical mouse and an APC UPS. It came with Windows XP which I never intended to use and my plan was to have Linux installed.

That was a time when high speed internet meant 64Kbps (Internet cafes proudly advertised that they had high speed ISDN lines at 64Kbps), Cable broadband meant 128Kbps and the only broadband operator was Sify who wouldn't offer me a connection because they didn't have cables running to my neighborhood. All these meant that download a Linux distro over the internet is nearly impossible. There were other other options though - A well known Linux magazine called "Linux For You" provide a companion CD, usually a bootable live CD with latest distros and other free software, and another option was the Local Linux User group http://ilugc.in where we used to maintain a directory with an index of all the CDs each member had so that we could borrow and take a copy. I lost count of all the distros that I had tried - Knoppix, Fedora, Debian, Gentoo, and numerous others. My PC made it an interesting proportion to try the various distros since none of them would work out of the box. GUI being the first bottleneck - The S3 Unichrome onboard GPU is not the mostly widely used that many wouldn't even have heard off. Obviously there were no graphics drivers so many would just fallback to the console, the ones that actually booted up to the GUI would use the vesa driver with a reduced resolution and a not so great GUI experience, then there was the multimedia keyboard that I had to figure the scan code and map them to the specific operations in the X keyboard layout configuration, Then there was this APC UPS which had a USB interface so I can hook it up to the PC and monitor the battery charging/draining status. Initially I had to compile the kernel modules to get this to working but sooner many distros started shipping all the necessary components to make it a  breeze.

Then came the "Warty Warthhog" - the first ever version of the Ubuntu. Canonical, the makers of Ubuntu had a nice program where I could order a set of CDs online and they would ship it to me for FREE. They encouraged to order a pack of CDs instead of singles so that I can give it to my friends and spread Linux. This was a totally nice and refreshing experience, where minimalism was the main theme. That was a Gnome based desktop and the whole desktop looked polished and fresh with an unusual brown theme. I loved the CD pouches with the Ubuntu logo. In fact they even shipped a few Ubuntu decals so that I can have them applied on my PC.

(to be continued...)

Flash memory Vs eMMC

Flash memory and eMMC are the two most common non-volatile memory found on embedded systems.

Lets start with Flash - There are two types of them:
1) NOR Flash - Byte addressable (Read a byte, Write a byte (only flip bit:1s to 0s but not vice versa), Erase a sector/block to set all the bits to 1)
2) NAND Flash - Block addressable (Read/Write one block - typically one block is read in to memory before individual bytes in the block can be accessed)

Both types are connected to the system address bus, i.e. they are memory mapped and can be accessed by referencing their memory address.

eMMC:
Embedded Multimedia Controller is basically Flash memory bundled along with a controller. The flash memory can be accessed only via the controller using the eMMC protocol.

Remote debugging a QNX application running on an ARM target

On the target

Make sure pdebug is running and listening on port 8000

On the host

Start the gdb

ntoarm-gdb

Connect to the target

target qnx remote-ip:8000

List all the process running on the target

info pidlist

Attach to a process (Use the above listing to find the pid of the required process)

attach pid

Debug the process

info / step / continue

Kill the process

kill

Detach from the process

detach

My shortlist of diesel cars

Looking to buy a diesel car with good safety features, comfort and a nice look.

My short list:

1. Fiat Punto - Emotion (~6.5 L on road chennai)

2. Volkswagen Polo - Comfortline (~6.5 L on road chennai)

3. Ford Figo - Titanium (~6 L on road chennai)

4. Maruti Ritz - ?

Tata motors/Fiat showroom, Ambattur:

I was impressed at the solid build of the Punto. One can feel the solidness just by touching the doors. The seats are comfortable and it's easy to get in to and out of the car - thanks to its height and ground clearance. I am very much impressed by this car but waiting to see the Polo before deciding. The only bad part is that sales and service are from Tata and I had a first hand bad experience when I walked in to the show room - There were no body to attend and 5-6 visitors who were looking at the cars. The booking amount is Rs.10K and it takes 15 days to deliver.

Volkswagen showroom, CIT nagar:

Polo will be available in showroom during 2nd week of march. 80 cars have already been booked in chennai. Petrol car delivery starts from April. One have to wait till june for the diesel engine. 50 K is the booking amount and it takes one month for delivery.

MPL Ford, Anna salai:

Figo is already available in showroom but one has to wait another week for a test drive. The 1.4L diesel engine is the same one from Fiesta. The top end model Titanium has airbags and ABS, but no rear power windows. It has an excellent boot space and good leg room at the back. The car doesn't look stylish though. 25K is the booking amount and it takes one month for delivery. Already 240 Figo's are booked in Chennai. (There is also a new MPL showroom near SRP Tools OMR)

Yet to visit a Maruti showroom.

Parsing and using custom extension in X.509 certificates

In the last two posts we saw how to create certificates with custom extensions and how to view extension in X.509 certificates, now it's time that we use them for some real purpose. The main purpose of placing custom extension is to express certain capabilities of the certificate holder. The receiving systems verifies the capabilities of the holder based on the presence of these extensions and the corresponding values in the extensions.

In the current example we will see the parsing of a non-standard extension called Admission with oid: 1.3.36.8.3.3 which is defined in the ISIS-MTT document

The ASN.1 description of the extension is given below

AdmissionSyntax ::= SEQUENCE {     admissionAuthority GeneralName OPTIONAL,     contentsOfAdmissions SEQUENCE OF Admissions } Admissions ::= SEQUENCE {     admissionAuthority [0] EXPLICIT GeneralName OPTIONAL,     namingAuthority [1] EXPLICIT NamingAuthority OPTIONAL,     professionInfos SEQUENCE OF ProfessionInfo } NamingAuthority ::= SEQUENCE {     namingAuthorityId OBJECT IDENTIFIER OPTIONAL,     namingAuthorityUrlQUENCE OF OBJECT IDENTIFIER OPTIONAL,     registrationNumber PrintableString (SIZE(1..128)) OPTIONAL,     addProfessionInfo OCTET STRING OPTIONAL } ProfessionInfo ::= SEQUENCE {     namingAuthority [0] EXPLICIT NamingAuthority OPTIONAL,     professionItems SEQUENCE OF DirectoryString (SIZE(1..128)),     professionOIDS SEQUENCE OF OBJECT IDENTIFIER,     registrationNumber PrintableString (SIZE(1..128)) OPTIONAL,     addProfessionInfo OCTET STRING OPTIONAL }

If we build a certificate with this extension and try to display the contents using the method described in the previous post, we will not be able to see the particulars of this extension nor will we be able to query any of the fields from this extension. This is because OpenSSL doesn't  yet know to parse the contents of the extension. To help OpenSSL in parsing the data we have to define the structure of the extension. Which is done as follows.

First, the needed data structures are defined:

typedef struct NamingAuthority_st {         ASN1_OBJECT* namingAuthorityId;         ASN1_IA5STRING* namingAuthorityUrl;         ASN1_STRING* namingAuthorityText; } NAMING_AUTHORITY; DECLARE_ASN1_ITEM(NAMING_AUTHORITY) typedef struct ProfessionInfo_st {         NAMING_AUTHORITY* namingAuthority;         STACK_OF(DIRECTORYSTRING)* professionItems;         STACK_OF(ASN1_OBJECT)* professionOIDs;         ASN1_PRINTABLESTRING* registrationNumber;         ASN1_OCTET_STRING* addProfessionInfo; } PROFESSION_INFO; DECLARE_ASN1_ITEM(PROFESSION_INFO) typedef struct Admissions_st {         GENERAL_NAME* admissionAuthority;         NAMING_AUTHORITY* namingAuthority;         STACK_OF(PROFESSION_INFO)* professionInfos; } ADMISSIONS; DECLARE_ASN1_ITEM(ADMISSIONS) typedef struct AdmissionSyntax_st {         GENERAL_NAME* admissionAuthority;         STACK_OF(ADMISSIONS)* contentsOfAdmissions; } ADMISSION_SYNTAX; DECLARE_ASN1_ITEM(ADMISSION_SYNTAX)

and then the C translation of ASN.1 representation.

ASN1_SEQUENCE(NAMING_AUTHORITY) = {         ASN1_OPT(NAMING_AUTHORITY, namingAuthorityId, ASN1_OBJECT),         ASN1_OPT(NAMING_AUTHORITY, namingAuthorityUrl, ASN1_IA5STRING),         ASN1_OPT(NAMING_AUTHORITY, namingAuthorityText, DIRECTORYSTRING), } ASN1_SEQUENCE_END(NAMING_AUTHORITY) ASN1_SEQUENCE(PROFESSION_INFO) = {         ASN1_EXP_OPT(PROFESSION_INFO, namingAuthority, NAMING_AUTHORITY, 0),         ASN1_SEQUENCE_OF(PROFESSION_INFO, professionItems, DIRECTORYSTRING),         ASN1_SEQUENCE_OF(PROFESSION_INFO, professionOIDs, ASN1_OBJECT),         ASN1_OPT(PROFESSION_INFO, registrationNumber, ASN1_PRINTABLESTRING),         ASN1_OPT(PROFESSION_INFO, addProfessionInfo, ASN1_OCTET_STRING), } ASN1_SEQUENCE_END(PROFESSION_INFO) ASN1_SEQUENCE(ADMISSIONS) = {         ASN1_EXP_OPT(ADMISSIONS, admissionAuthority, GENERAL_NAME, 0),         ASN1_EXP_OPT(ADMISSIONS, namingAuthority, NAMING_AUTHORITY, 1),         ASN1_SEQUENCE_OF(ADMISSIONS, professionInfos, PROFESSION_INFO), } ASN1_SEQUENCE_END(ADMISSIONS) ASN1_SEQUENCE(ADMISSION_SYNTAX) = {         ASN1_OPT(ADMISSION_SYNTAX, admissionAuthority, GENERAL_NAME),         ASN1_SEQUENCE_OF(ADMISSION_SYNTAX, contentsOfAdmissions, ADMISSIONS), } ASN1_SEQUENCE_END(ADMISSION_SYNTAX)

Once this is done we have to some how say about our new extension to openssl:

static X509V3_EXT_METHOD ext_admission = {         .ext_nid = 0,         .ext_flags = 0,         .it = ASN1_ITEM_ref(ADMISSION_SYNTAX),         .i2s = NULL,         .s2i = NULL,         .i2v = NULL,         .v2i = NULL,         .r2i = NULL,         .i2r = i2r_AdmissionSyntax, }; /*  * Tell about our new extension to OpenSSL  */ void x509_add_custom_extensions() {         ext_admission.ext_nid                  = OBJ_create("1.3.36.8.3.3", "Admission", "Admission");         X509V3_EXT_add(&ext_admission); }

Now if we have an extension we can convert it to the corresponding data structure and then play with fields / verify them.

ADMISSION_SYNTAX* x = (ADMISSION_SYNTAX*) X509V3_EXT_d2i(ext); sk_num(x->contentsOfAdmissions) sk_value(x->contentsOfAdmissions, iAdmission);

i2r_AdmissionSyntax is a simple function (that you may write/or set to NULL) which converts the internal data structure to some human understandable form and is used when the function X509V3_EXT_print_fp is called.

Viewing extensions in X.509 certificates

In the last post we saw how to create certificates with custom extensions - as a second step let us see how we can access these extensions and make sense of them. The code below opens a certificate, counts the number of extensions in it and iterates over every extension and prints a representation of the extension understandable to, us, humans.

X509V3_add_standard_extensions(); inf = fopen("mycrt.crt", "r"); cert = (X509*) PEM_read_X509(inf, NULL, NULL) count = X509_get_ext_count(cert); for(i = 0; i < count; i++) {         ext = X509_get_ext(cert, i);         printf("%s\n", OBJ_nid2ln(OBJ_obj2nid(ext->object)));         if(!X509V3_EXT_print_fp(stdout, ext, 0, 0)) {                 ERR_print_errors_fp(stderr);         }         printf("\n");         X509_EXTENSION_free(ext); }

It doesn't print the human representation of all the extensions found but only for the built in extensions, because the library doesn't yet know to represent the custom extensions that we have placed in the certificate.

More about parsing custom extensions and making sense of the values in it in a later post.

Creating X.509 certificates with custom extensions

Every time when I need to do something with OpenSSL, it involves searching lot of places including the library code itself to achieve my target - This time the requirement is to generate a X509 v3 certificate which contains non-standard/custom extensions. Luckily I found convincing documents early enough.

Here the task is to generate a CA certificate with standard extensions and then to create another certificate containing custom extensions and sign it with the newly created CA. The code below achieves both of the tasks.

1. To generate the ca certificate, run the script as ./genkey.sh myca ==> which generates myca.key and myca.crt
2. To generate the authentication certificate, run the script as ./genkey.sh myca mycrt ==> which generates mycrt.key and mycrt.crt and signs it with myca.key

The script is mostly self explanatory and contains an inline openssl config file - The main points to note are:
1. Extension are to be placed separately in a named section
2. Custom extension are of the form oid=DER:<DER-Encode-Hex-Values>
3. openssl command x509 should be given the extension file name and section name

#! /bin/sh # Filename: genkey.sh if [ $# -lt 1 ]; then         echo "Usage: $0 ca-name [new-cert]"         exit -1 fi ca="$1" new=${ca} if [ $# -ge 2 ]; then         new="$2" fi if [ -f ${new}.key ]; then         echo "${new} already exists: Delete ${new}.key & ${new}.crt to proceed"         exit -1 fi #----[ inline config file ]-------------------------------------------- ca_extensions="ca_extensions" cert_extensions="cert_extensions" config=.${new}.config cat > ${config} << EOF [ req ] default_bits           = 2048 default_keyfile        = ${new}.key distinguished_name     = req_distinguished_name attributes             = req_attributes prompt                 = no [ req_distinguished_name ] CN                     = ${new} OU                     = ouTest O                      = oTest C                      = IN [ req_attributes ] [ ${ca_extensions} ] basicConstraints=critical,CA:true subjectKeyIdentifier=hash keyUsage=keyCertSign,cRLSign authorityInfoAccess=OCSP;URI:http://ocsp.test.com:8080/ [ ${cert_extensions} ] subjectKeyIdentifier=hash keyUsage=critical,digitalSignature,keyEncipherment certificatePolicies=1.2.276.0.76.4.64 crlDistributionPoints=URI:ldap://ocsp.test.com:389/cn=test Komponenten Testreferenz CA01 authorityInfoAccess=OCSP;URI:http://ocsp.test.com:8080/CMOCSP/OCSP authorityKeyIdentifier=keyid 1.3.36.8.3.3=DER:\ 304DA421301F310B30090603550406130244453110300E060355\ 040A130767656D6174696B302830263024302230150C13416E77\ 656E64756E67736B6F6E6E656B746F72300906072A8214004C0477 extendedKeyUsage=clientAuth,serverAuth EOF #----[ Generate key and sign ]------------------------------------------ # 1. Generate Key & CSR openssl req -new -nodes -config ${config} > ${new}.csr # 2. Self sign (or) Sign with CA if [ ${new} == ${ca} ]; then         echo "Generating self signed CA: ${ca}.crt"         openssl x509 -extfile ${config} -extensions ${ca_extensions} \                 -sha1 -req -signkey ${ca}.key < ${new}.csr > ${new}.crt else         echo "Signing ${new}.crt with ${ca}.crt"         openssl x509 -extfile ${config} -extensions ${cert_extensions} \                 -sha1 -req -CAkey ${ca}.key -CA ${ca}.crt < ${new}.csr > ${new}.crt fi rm ${new}.csr rm ${config}

Resolutions for the year and after

1. To overcome my laziness.
2. To be more focussed and ignore distraction.

Top two things needed to shape myself in to a better person.
Hopefully I will be able to overpower laziness and distraction.

Bypassing Websense

Websense is a kind of proxy server that filters request to the internet and will block certain kinds of request - The most common way of blocking is to check the requested url against a black list.
The most common HTTP request is GET and looks like:

GET /path-to-file HTTP/1.0
Simply replacing this with something like the one below seems to confuse Websense, making it to believe that the url is not in the black list.

GET /path-to-file <2048-space> HTTP/1.0
While some web servers are also confused by such a request some others behave normally.

To change the request from the first format to the second one needs some one in between and the easiest is a proxy server and if it's in python it only gets more easier.
TIP: If using windows, the extension of the proxy file can be set as .pyw to run it invisibly.

The browser has to be configured to use a proxy but since some sites doesn't work with this proxy due to the request mangling, it is worth writing a proxy.pac file that uses the proxy only for certain sites.
QuickProxy is a nice add-on for Firefox that makes it easy to switch proxy configuration.

Yet another revival

Time and again I try to keep my blog alive but after a few posts the interest fades away mainly due to the effort involved in posting. I have tried different tools but none could stand my laziness - but this time posting via email seems to be easy enough to carry forward my lazy ass.

Here it is - Google Chrome

It's here to flame up the fire and I am exploring it right after waking up.

The first thing that can be noticed is the location of tabs, the tabs are located above the url box - It's like different independent browser windows joined together rather than a single window containing different web pages.

The URL is colour coded - The server name appears dark and the rest of the path appears in grey. https is highlighted in green. A tiny thing but it's much easier to identify the server name.


By default there are two processes while opening Chrome. The process count increases on creating the first new tab but subsequent new tabs doesn't increase the process count - The count increases only on loading a page.


Killing a process displays a message in the tab corresponding to that process and doesn't kill the entire browser - Good.

Very spacious - Tabs are merged with the title bar, no status bar, no menu bar.


Acid test results:



Only a boot strap is available for download which in turn downloads and installs the actual browser - This possibly may be for region specific customization but I had prefer the real installer.


Missing:
- Profiles
- Addons
- Couldn't find offline mode

@Me, should find more time to explore