Digital Signal Processing and Discrete Fourier Transforms – the start of my journey…

Written as a retrospective…

A bug that I had picked up was to remove what was described to me as ‘high frequency ringing’ in the output of a derivative (determining the rate of change between recorded values) calculation, this calculation is performed in the frequency domain.

To add a layer of complexity:  The input signal is the output of another process.  This other process can have a mask applied to it, reducing the amount of data is calculated.  This means that there can be a block of data in the middle of our signal, and then rest of the signal has zero value on either side of the data.  You could imagine that this block of data has ‘hard edges’.  As there is data in the middle of signal, we don’t expect to see results that are longer \ outside of the input data, but that’s exactly what can we have.

Our resulting data looks horrible, with artefacts in it.  Not good.

There are two issues at play here.

  1. The ringing going past the input data edges when it has a mask applied
  2. High frequencies being boosted within the signal

And the solutions?

  1. Find where the data starts and ends, and mirror both ends of the signal
  2. Apply a filtering on a vector that relates to frequencies, this vector is used in the derivative calculation

Solutions in some more detail

The beginning, the end, and mirrors

As we can have data in the middle of a signal, which contains hard edges, we want to taper our signal.  But we don’t want to taper real parts of the signal, otherwise we have an untrue result.  Finding the start and end points are easy; iterate through the signal and record the first and last non-zero value.

For the mirroring, we need to know how much of the data is going to be tapered.  When we know what percentage of the signal will be tapered, we can determine how long to make our mirrors so that the leading taper stops at the start of our real signal, and the trailing taper starts at the end of our real signal.

The the results are then manipulated to reconstruct an output signal with the same data beginning and end points, and we then don’t have the high frequencies shooting up and down from where the real data is.

Filtering out the frequencies

When a derivative is performed in the frequency domain, it dampens the amplitude of the lower frequencies and boosts the amplitude of the higher frequencies.

We don’t want that to happen, hence the filtering.

Firstly, we want to determine from what frequency on wards should be attenuated, then we work backwards. As an example, I don’t want to see frequencies greater than 80Hz.  I first need to find the index in the vector that corresponds¹ to 80Hz, decide on some kind of tapering length, and start tapering from index of 80Hz – tapering length.

The taper is a trailing taper, we used cosine squared, to have a nice effect.
Bringing these two techniques together results in much cleaner results when visualised.

Bringing the solutions together

The solutions are neatly coded up into nice modular functions, with unit testing.  It’s all integrated into the software, allowing for interactive filtering \ attenuating of a user specified frequency if the end user wants to have filtering turned on.  It’s resulted in a pretty slick experience for the user, something I’m proud of being able to produce.

Summary

I admit that this all sounds high level stuff, but that’s because I don’t really understand the maths involved with tapering, how the derivative is being performed in the frequency domain, etc…  I’ve had plenty of help from my clever Mathematician colleagues, plus I’ve read a lot of the code where there is pre-existing functionality.

I’ve not even touched upon what needs doing to the data before the derivative calculation is performed; there’s preparation needed on the data before Fourier Transforms are executed.  Nor the testing and prototyping involved; producing tests based upon analytical signals with known derivatives, testing calculating index in the vector of a given frequency, etc…

This has been a really good learning curve in some very interesting topics, aided by this fantastic resource: https://jackschaedler.github.io/circles-sines-signals/

I’ve not really done this post justice by omitting before and after examples.

¹ Or, as close to, if I’m being lazy and not using some kind of interpolation – which in fairness interpolation has not been implemented.

Unpicking the puzzle

Finding out and understanding how all the bits fit together

Imagine having a completed puzzle in front of you. The puzzle is so well cut out that you can’t see the individual pieces from a distance. Depending on how you angle yourself looking at the puzzle, you can see certain chunks. Change your angle and distance again, you’ll see other sections but lose sight of previously seen segments. Go so close that your nose is touching, you then become consumed by the intricacies of all the little connecting pieces, completely loosing your self in what fills your vision. How do you go about understanding how this was made?

I started my career as a software engineer in what is probably a really bad, really risky way; joining a group with zero engineering experience and some very minimal knowledge on writing code and jumping straight into a seriously complex code base with no resources on this code base. Scary stuff, right. Sink or swim springs to mind…
A lot of my role is marred by but not having access to a design or architectural overview, unit tests or the commit history. Sensibly speaking, this is a serious challenge and I suspect could potentially put people off as a risky working environment. I take that risk and make it a positive opportunity to learn, learn to read code and not be intimidated by lack of understanding.
This lack of design means that what ever feature or functionality that the client wants, there is an amount time that is taken to investigate what there currently is and how what is requested could work; multiple implementation options and what the pros, cons, limitations, risks and future possibilities of each of these options are. This investigating stage is realised by reading code, a lot of code, and putting together what could be some form of UML (for want of a better description – it looks like UML but I’ve not taken the time for learn UML and follow it’s rules) and picturing how the components of what I’m look at fit together, their relationships.
Don’t get me wrong, I do run and use the application in debug with break points set where I think I should see something happen to help me understand and\or prove my thoughts. Using debug also helps in determining a flow to a process.
I think this results in accelerating a skill of reading code and how all the bits and pieces fit together. Admittedly this skill is being used on one code base, though I believe it is a skill that I can apply to any other code base that I would come across.

Apart from understanding what happens in the code base, how else do I make use of this skill? Fixing bugs!
A user reported an intermittent, inconsistent state of part of the application when starting the application and loading their data; this part that is inconsistent is to do with input validation. Toggle an input everything would be ok.
Great, that sounds like a pain to work out what’s going on. To begin troubleshooting, there are questions to be answered:

  • What are the factors involved when it works and when it does not?
  • Why does toggling the input make everything ok?
  • Do we have examples where things are working consistently?
  • Do we have other examples of inconsistent behaviour?
  • Why is it invalid and is that actually the case – are we experiencing some kind of latent behaviour that is not cleared?

As examples of working cases are understood, we see that the implementation in the flakey version is not consistent with the working example. We can see that bits are missing, or in the wrong place, or not even used! A plan is devised, some tests are written, headphones are put on and fixing commences.
The plan has typically been a case of re-writing portions of the code so that they now behave in the desired behaviour, other parts are refactored for clarity, some parts are just removed, deleted, got rid of. End result is an understanding of what is happening in a certain flow of the application, an understanding of how it should work and what different implementation of the same behaviour could look like. An internal (development team) by-product is unit tests, where applicable, and some drawings (my made up UML, CRC type cards, UI wire framing and what the elements are connected to and what they’re behaviour is) and documentation. Why? It helps the team out and mitigates the bus factor.
Another way I use this skill is for the implementation of a feature. Is there existing code I can use to do what I need, and how do I implement features and functionality whilst trying to adhere to the current implementation of the design, using as much pre-existing abstraction as possible.
The skill is used as I work daily, as I develop application framework, when I decide if the abstracted method is suitable or if I need to override it or do something different.  If there is not something suitable, do I now need to design something new, and if I do how can I do it so that it feels like it’s part of the code base and not just tagged on.

In principle, this sounds simple. In reality, I feel that this is a disciplined approach to working; it takes time, energy and practice. It can be tough to read over the same block of code time and time again and feel no closer to an understanding of it’s role.
It has strong similarities to learning; observe and then do. At school we didn’t go straight into creative writing in our English lessons, we read stories, and of different genres, before we wrote our story.
I don’t know if every software engineer works in this manner, I’ve not asked my peers, though it feels like a good way to describe vocationally what I do and how I do it when asked by new people.
I feel empowered to have developed this skill quite early on in my career, so much so, I feel it’s my greatest skill; I feel confident to be able to work my way through a code base and make a contribution.

Going back to the sink or swim situation, I think I’m surfing it…

A UnitOfMeasurement converter for screen capturing for print

A recent feature I was task with was to implement a unit of measurement converter; inches to pixels to mm.

The context of the feature is that the application has a capture (image) to print option, where the output was specified in number of pixels with a label/text that showed what this would be in inches.  Not useful.

The feature requirements were:

  • Choose a unit of measurement as the desired output from a drop-down selection of centimeters, millimeters, inches and of course, pixels.  I don’t feel that using pixels as a unit of measurement for print made much sense, but hey, it stuck :-\
  • Display the output size in all units beneath the input fields.
  • The input fields to be kept up to date with the correct value when the selected unit was changed.
  • Not to mess with the aspect ratio mechanism in place.
  • The output mechanism should not be changed; it still used pixel value to create the file.Not a massive task, yet enough for a few days work.

The class to do the conversion I decided to be static inner of the screen capturing class.  The class needed to know what unit of measurement was selected from the drop down and provide that output as an output for all the other units.  An initial draft of the class put together and then dived into writing some tests for each unit of measurement with known inputs and the expected output; expected outputs worked out on paper with the formula documented in the unit test; that’s an acceptable comment block 😉

Tests diligently created ensuring that each unit will convert to the other units as expected, as well as what happens when the selected unit then changes.  I also even thought at this stage that pixels have to be rounded to a whole number, and put in a test to demonstrate!  <That would have introduced a spanner in the works when that case was encountered by a user…>

Now the test class was constructed, time to build up the actual converter.
The converter class was actually an Enum as its methods were static giving a return value, each unit of measurement didn’t need to be instantiated.  I only needed to know which unit I was using and what the input values for that unit were.  The combobox was populated using ComboBoxModel which would return the Enum associated to that entry, then firing the necessary conversions and updating of output sizes text.

Converter built and tests passing, now time to tie it into the existing UI along with the aspect ratio listener.
Integrating the combo box and additional output size labels took a couple of goes at amending the JGoodies constructs, once accomplished it looked fairly neat.

Time to take the new feature for a spin!  Fired the program, got a component ready so I can capture it for print, start the amended UI and select an unit.  Fiddle with the values and see that the output sizes update accordingly (they do \o/ :-)).  Now… the actual test of writing an image.  And it works.  Initial pass of the feature completed and it works as expected.  Time to tidy it up, refactor and push for a review.

Something that I feel that could be made better was that the screen dpi was set as a fixed value.  This doesn’t feel right as screens have different dpi’s with the desktop manager having to provide a satisfactory image to be displayed on the screen.  Querying the hardware for what the dpi of the hardware I think would be safer for producing the output image files.  That said, it was working to generate files with the specified output size before I implemented the converter; some magic going on here (a.k.a. I don’t understand just yet).

public enum UnitOfMeasurement {

Pixels("Pixels", 1),
Millimeters("Millimeters", 25.4),
Centimeters("Centimeters", Millimeters.outputScale / 10),
Inches("Inches", 1);

 private String name;
 private double outputScale;

 UnitOfMeasurement(String name, double scale) {
  this.name = name;
  this.outputScale = scale;
 }

 double getValueInInches(double enteredSize) {
 double result = 0;

 switch(this.name) {
  case PIXELS:
   result = (enteredSize / printDPI.get());
  break;
  case MM:
  case CM:
   result = (enteredSize / outputScale);
  break;
  case IN:
   result = enteredSize;
  break;
 }

 return result;
 }

double getValueInPixels(double enteredSize) {
 double result = 0;

switch(this.name) {
 case PIXELS:
 result = enteredSize;
 break;
 case MM:
 case CM:
 result = (enteredSize / outputScale * printDPI.get());
 break;
 case IN:
 result = enteredSize * printDPI.get();
 break;
 }

return Math.round(result); // This is a correct method to call
 }

double getValueInMM(double enteredSize) {
 double result = 0;

switch(this.name) {
 case PIXELS:
 result = enteredSize * Millimeters.outputScale / printDPI.get();
 break;
 case MM:
 result = enteredSize;
 break;
 case CM:
 result = enteredSize * 10;
 break;
 case IN:
 result = enteredSize * Millimeters.outputScale;
 break;

}
 return result;
 }

double getNativeValueFromPixels(double pixelSize) {
 double result = 0;

 switch(this.name) {
  case PIXELS:
   result = Math.round(pixelSize); // This is a correct method to call
  break;
  case MM:
  case CM:
   result = pixelSize * outputScale / printDPI.get();
  break;
  case IN:
   result = pixelSize / printDPI.get();
  break;
  }
  return result;
 }

 private static final String PIXELS = "Pixels";
 private static final String MM = "Millimeters";
 private static final String CM = "Centimeters";
 private static final String IN = "Inches";

}

Remote collaborative working

comparison of remote pair programming tools

The majority of my colleagues are not based in the same office, or even country, so to practice \ do pair programming we need to do some form of screen sharing. We have used webex, which was working well enough until it stopped working due to some technical issue on Gentoo (Cisco not supporting webex on 64bit OS and browser).

Trying to get it to work was not successful, so we looked for alternatives.  The tools we tried out were:

  • Floobits IntelliJ idea plugin
  • HP RGS

I’ll try and produce a comparison of each of these collaborative tools as I’ve experienced them.

Webex

Webex is a web conferencing tool owned and developed by Cisco.  You have to have a webex account and I don’t think that it’s free to use; I’m using it via work, so I have an account with my work email address.

A 32bit browser and 32bit Java plugin is required to use webex.  It opens an applet where you can have whiteboard and participant chat window.  The feature I’ve mainly used is the Share Desktop tool.  This will share the entire of your desktop to who joins the meeting, this includes all running applications and even notifications.  This can be good and bad.  The good is that when you run ‘stuff’, it is all seen, including error messages and the state of what ever it is that you’re demonstrating; so this will give an accurate depiction of what is happening.

The not good bit is that you don’t control what your participants see when you share the entire desktop; IM’s, emails, etc.

When sharing the entire desktop, the resolution of the screen(s) is sent as is, if the participants have a smaller resolution that you do, they will have a window with scrolly bars and have to do some jigging around of window positioning.  This is ok when it’s one person that has joined the conference, but may not be great when a number of people join who have a smaller resolution that the host.

It is possible to choose an application to share, so only that application is displayed to the participants.  I’ve not used this feature much at all, so can’t comment on it other than, it worked.

One thing that webex lacks is support for application key shortcuts, these are just not sent.  I like command keybindings to do stuff quickly and use the mouse as little as possible.

The usage scenario is that one person will schedule the web conference and send out a link, people join the conference.  A voice conference call is up at the same time too.  The conference initiator \ host is usually the one who shares their desktop \ application and with the people on the phone viewing, commenting and discussing what’s to be done.

When it worked, it was good enough.

Floobits

Floobits is a an IDE plugin that allows code to be edited by multiple people at the same time in a session \ workspace.  Again, you need some kind of accounts on with floobits and create workspaces (repositories) and upload code to it in a public or private workspace (repository).

We used floobits on a small, non-work coding example to test it out.  Zero chance of putting code onto a cloud service, I like things to be within the firewall.  Saying that, they plan to offer a within the firewall solution and other plans with a number of private workpaces.  It can integrate with GitHub too.

Once accounts and workspaces were created, we created a collaborative session and got to work.

Because it’s an IDE plugin, keybindings work :-).  We could both open multiple files and edit these files at the same time.  We could even both edit the same file at the same time.  We can watch changes that the other is doing.  We can independently run the app, and tests and in debugger too; providing that a change doesn’t stop the code from compiling.  We didn’t test to see what would happen with revision control.

One big positive we found is that we could discuss what needed doing and do these things in parallel if needed.  Alternatively, we could stick to typical pair programming and discuss one concept at a time, but with our own resources independently open; documentation, manuals, etc.

We stuck to talking on the phone, however floobits can integrate with google hangouts, so voice and video can be done in a hangouts group.

We didn’t test with more than two people working at the same time.

We also encountered some filesystem inconsistencies, but I think it’s because we didn’t really explore who ‘hosts’ the workspace and what connecting collaborators should do in regards to overwriting files.

I really liked floobits, it was a pretty neat experience, however I don’t believe the company will pay for it at the moment.

HP Remote Graphics Software

HP RGS was designed for remote high quality graphics, I believe.  It works in a client \ server set up that they term sender (server) and receiver (client).  The sender is an X server plugin, making it available when the X server starts up.  The receiver connects to the host over it’s propriety  protocol and codecs for compressing the data being sent.

The sender is a licensed product whereas the receiver is not, but, with Z series HP workstations, you don’t need to pay.  We have Z series workstations 🙂

I won’t go into the configuration of the sender here.  The receiver prompts for a hostname to connect to, credentials are then prompted for and voila, desktop is shared.

RGS sends the entire desktop, but it can try and match resolution and layout.  The receiver can be made to be a collaborator, so they can take control of the mouse and keyboard.  Keybindings worked in the IDE for the collaborator \ receiver, which is pretty handy.

As the whole desktop is shared, this has the same pitfalls as mentioned for webex, I found the overall experience is better than webex; no additional windows and dialogues cluttering the desktop.  The receiver can set the receiver to be full screen.  The image quality can be adjusted depending on the network bandwidth and latency; however, running in the LAN \ WAN, we used full quality which was good – again, better than webex.

Again, we have a phone call going to talk.

A couple of  negative aspects… Presentation can’t be given to someone else.  To achieve this, they would have to have RGS installed and then you would need to connect to them, this isn’t that serious though.  If you’re remoting, you’re dependent on the bandwidth, latency and stability of your connection, you can loose connectivity and experience some lag and connectivity warnings.

The experience was pretty good using RGS.

Conclusion

I’m glad that webex broke for my colleague so that we were forced to find alternatives.  As much as I liked using floobits, I think RGS wins.  Reasons being, everything remains in the firewall, shared image quality is very good (very nearly native, or I can’t tell that it isn’t native), pretty stable, no need for external accounts as AD authentication is used, we don’t have to pay for it as the license is bundled with the hardware (yes, money is a factor and I suspect it will not be available if the IT people decide to change the hardware purchased).

Regression Trend lines – how it was done

An update on this feature.

It’s mainly done, implemented as linear trend where the line can be edited by moving the end points of a chosen line.

The hardest part of this feature has been deciphering how it was all pieced together in the first place.  From this deciphering came what we wanted to replicate, to a degree; there was already a functionality where a polygon could be drawn on the chart area, where the polygon’s vertices would be saved to a session database so the object could be reloaded when the user resumes the session.

The functionality that was used as the basis of rendering a line on the chart area was analysed into what could be common with the a line.  Once determined, an amount of abstraction was done.  This abstracting was done so that the class that is responsible for drawing the polygons did not have to be changed too much, keeping as much of the previous design in place.

Other aspects that were worked on were:

  • Displaying the line model equation; slope, intercept and R^2
  • Choosing a library to perform the regression calculation, we settled on Apache Maths
  • Ensuring that the line remains within the chart area when it is rendered.  We implemented Cohen-Sutherland line clipping algorithm
  • The functionality of editing a line.  This meant we had an edit mode for the line model were the line has handles that are clicked on and moved within the chart area.  Once the line model editing has finished, then the slope, intercept and R^2 of the line are recalculated
  • Saving the properties of the line model(s) into a session file \ database and loading them when necessary
  • Extracting the required data for passing into the Regression class.
    • The raw data undergoes a randomised (from a unique seed to ensure it’s randomised in the same way all the time) decimation and is then binned to limit the number of points to 63,000 data points
    • The data extracted from this binning process needed to be weighted again to yield the correct regression calculation results
  • Writing unit tests

There are some things I would like that we could have worked more on.  One thing would be to amend how the shapes are rendered on the chart area.  As there are multiple polygons and lines that could be rendered on the chart area, they are held in a List<>.  When it came to editing one of these shapes, a copy of the shape is made rather than directly editing the shape in question.  This doesn’t feel like great design \ implementation of this functionality.  Because a copy of the shape that is being edited is made, it manifests itself as a visual ‘fault’.  If there are more than one generated line models, as opposed to edited line models, and one of these is then chosen to be edited, the line that is currently being edited can look to disappear if it sits above another line.
I would also like to have ensured line clipping on the polygons, as they could leak out of the chart area.

If desired, I may be tasked with putting in other regression models.

Something new! Regression \ trend lines

A small, work related, learning project that I’m playing with: https://github.com/sebastientorres/RegressionLines.

It’s a practical example of something that I will have to work on in the near(ish) future to produce some 2DGraphics of trend lines on a cross plot.  A couple of requirements are that the trend line could be edited for (human eye) visual best fit and for contour type heat mapping, if there was a third attribute available on what is being plotted; density, some other value range, something else in common.

Unfortunately, this is all that you will get see of the efforts.  I don’t think that I will be able to give a code example of what will be done at work, which will be substantially more complex…

It’s a work in progress…

“Happy Coding”
– John Purcell, CaveOfProgramming.com

CPU Accounting on linux

In my new role, I was tasked to produce some reporting on a process that a user was running on their workstation for client billing purposes.  This meant turning on accounting on the workstation (we use CentOS 6.x).

This is all very well and good, however we need to be able to parse the file(s) to get the information that we want, thus I got to work with a bit of perl to process the files and produce a report.

The script takes the following arguments:

  1. directory where the files reside
  2. reporting period start date
  3. reporting period end date
  4. regex\string name of the process to search for

My idea behind this is that there is a cron job on the workstation that will copy the pacct-*.gz files from /var/account to somewhere out of the root filesystem and onto another filesystem local to the workstation (not an nfs mount, but local disk that is not on the root logical volume).
The files will accumulate, which gives the flexibility on reporting date ranges and gives us a historic view of what the workstation is doing.

It’s pretty rough and does the job, I’m sure there are some things that could be done to improve it, use it if you want – I give no warranty on it’s performance and affect on a system, it worked perfectly well on the systems that I tested with at work…

Script is on github: https://github.com/sebastientorres/SysAdmin/blob/master/cpuAccounting.pl

Windows Terminal Services and an rdesktop wrapper

We’re implementing Windows Terminal Services farm with session brokering.  This kind of gives me a reason to redevelop the current wrapper script that parses the options and values to rdesktop for launching the session.

There is a user experience based reason as to why look at the script again.  When the login session is started, the user has to enter their password into the rdesktop session, if the session is redirected then the password has to be entered again; this is not a seamless experience so I dislike it.
The solution is to give the password to rdesktop from within the wrapper script, when this happens the session redirect experience is seamless; that’s what we want.

So my envisaged solution is a gui where the username is prepopulated (reduces user error) and the password is entered, hitting the login button reads in the entered password (sanitised, of course) and executes rdesktop.
Why a gui?  I wanted to give something more to the user experience.  Currently it’s possible to choose a resolution, but I think that this is a little known feature, so it would be nice to make this more apparent to the end user.

Code to be posted\linked upon completion.

Update:
Unfortunately I didn’t finish this before my time was up in this role, however I will post it somewhere when I dig it out; a lot of tidying up was done before changing role.

Headless Xorg configuration

Every now and then you need a graphically fully functioning Xorg running with a pretty decent resolution, 2560×1440 with a 24bit depth, for example.  The machine needs to be headless too, to save space.

To achieve this, Xorg needs to have a static configuration, which needs (well, took me) some trial and error.  The hardware this was run on was a HP Zseries workstation with an Nvidia Quadro K4000. it also worked with a GeForce GTX580.

Minimal xorg.conf with debug options

As we have an nvidia card, a minical xorg.conf was generated using the command ‘nvidia-xconfig’.
Then edit this file and add the following line in the device section:

Option    "ModeDebug" "true"

The device section relates to the video card in the machine.
Restart X to pick up the new config

init 3 && init 5
Compiling the xorg.conf

Using the amended minimal driver generated config file, it will fail.  This is where the trial and error comes into play and checking with the Xorg log file, /var/log/Xorg.0.log.

The steps to take in putting the final file together are:

  • Amend the ‘Device section
  • Choose a resolution and run it through cvt
  • Amend the ‘Monitor’ section
  • Bring it together in the ‘Screen’ section
  • Try it!

Performing the listed will result in ‘tricking’ Xorg that it has some hardware attached to it so that it can run.
Before we start making an entry for these sections, we want to find out from the failed xorg config what outputs the video card has; we less Xorg.0.log.
To start with, the line we’re looking for is:

"NVIDIA(0): Valid display device(s) on Quadro 4000

Then the lines beneath it that states what channels \ outputs are functional and what the maximum pixel clock speed it supports is:

NVIDIA(0): Valid display device(s) on Quadro 4000 at PCI:5:0:0
NVIDIA(0):     CRT-0
NVIDIA(0):     Lenovo Group Limited (DFP-0) (connected)
NVIDIA(0):     DFP-1
NVIDIA(0):     DFP-2
NVIDIA(0):     DFP-3
NVIDIA(0):     DFP-4
NVIDIA(0): CRT-0: 400.0 MHz maximum pixel clock
NVIDIA(0): Lenovo Group Limited (DFP-0): 330.0 MHz maximum pixel clock
NVIDIA(0): Lenovo Group Limited (DFP-0): Internal Dual Link TMDS
NVIDIA(0): DFP-1: 165.0 MHz maximum pixel clock
NVIDIA(0): DFP-1: Internal Single Link TMDS
NVIDIA(0): DFP-2: 165.0 MHz maximum pixel clock
NVIDIA(0): DFP-2: Internal Single Link TMDS
NVIDIA(0): DFP-3: 480.0 MHz maximum pixel clock
NVIDIA(0): DFP-3: Internal DisplayPort
NVIDIA(0): DFP-4: 480.0 MHz maximum pixel clock
NVIDIA(0): DFP-4: Internal DisplayPort

I chose an output with a maximum clock pixel Mhz rating, such as the Internal DisplayPort DFP-3 or 4; higher clock speed suspect higher resolution.

Now we’re at the point where we can put together the device section.

We want X to to think we have a monitor attached to the card but to not query it, thus the important lines to have in the Device section are:

Options "ConnectedMonitor" "DFP-3"
Options "UseEDID" "false"
Options "UseEdidFreqs" "false"

That’s the first part of the magic, next part is the Monitor section.

In the Monitor section we define what the the supported resolutions are using the line Modeline.  DFP-3 comes into play in the line ModelName.  The horizonal sync rate and the vertical refresh rate are also set.  UseEDID and UseEdidFreqs are again set to false.
Setting the Modeline and horizontal refresh rate, the command cvt is used to generate the values for the line.  Example usage is:

# cvt 2560 1440 60

Which will generate something like:

# 2560x1440 59.96 Hz (CVT 3.69M9) hsync: 89.52 kHz; pclk: 312.25 MHz
Modeline "2560x1440_60.00" 312.25 2560 2752 3024 3488 1440 1443 1448 1493 -hsync +vsync

Note the beginning of the first line it gives the resolution and then the value of 59.96 Hz, this is the vertical refresh rate.  Further across that line there is the value for the horizontal refresh rate, listed after hsync (89.52).
The second line is what is entered in the Monitor section.

Now it’s all brought together in the Screen section.
Device line refers to the identifier of the Device section (video card) identifier, Monitor line refers to the Monitor section identifier.

In my working xorg.conf, I have some other lines set (you’ll see them listed in the example below).

And that’s it.  Reboot the machine (required for X picking up the connected monitor) and fingers crossed, it’s running.  You can check that X is running by looking at the end of /var/log/Xorg.0.log and the string ‘server terminated’ is not there at the end of the file.

Enjoy.

Final xorg.conf file
# Generated by: Sebastien.Torres
# Headless configuration giving a resolution of 2560x1440 (27" wide screen) with the idea being that this machine runs RGS and someone connects to it

Section "ServerLayout"
 Identifier     "Layout0"
 Screen      0  "Screen0"
 Option      "Xinerama" "0"
EndSection

Section "Files"
EndSection

Section "Module"
# HP Remote Graphics Extension
 Load           "rge"
 # extmod recommended for preventing XID overflow.
 Load           "extmod"
 # Ensure X.org defaults are loaded
 Load           "glx"
 Load           "dbe"
 Load           "record"
EndSection
Section "InputDevice"
 # generated from data in "/etc/sysconfig/keyboard"
 Identifier     "Keyboard0"
 Driver         "kbd"
 Option         "XkbLayout" "gb"
 Option         "XkbModel" "pc105"
EndSection

Section "InputDevice"
 # generated from default
 Identifier     "Mouse0"
 Driver         "mouse"
 Option         "Protocol" "auto"
 Option         "Device" "/dev/input/mice"
 Option         "Emulate3Buttons" "no"
 Option         "ZAxisMapping" "4 5"
EndSection

Section "Monitor"
 Identifier     "Monitor0"
 VendorName     "Unknown"
 ModelName      "DFP-3"
 HorizSync   89.52
 VertRefresh 60
 Option      "UseEDID" "false"
 Option "UseEdidFreqs" "0"
 Option "ExactModeTimingsDVI" "true"
 Modeline "2560x1440R"  241.50  2560 2608 2640 2720  1440 1443 1448 1481 +hsync -vsync
EndSection

Section "Device"
 Identifier     "Device1"
 Driver         "nvidia"
 BoardName      "Quadro Q4000"
 Option         "ConnectedMonitor" "DFP-3"
 Option         "UseEDID" "false"
 Option         "UseEdidFreqs" "false"
EndSection

Section "Screen"
 Identifier     "Screen0"
 Device         "Device1"
 Monitor        "Monitor0"
 DefaultDepth    24
 Option         "UseEDID" "FALSE"
 Option "ExactModeTimingsDVI" "TRUE
 SubSection     "Display"
   Depth       24
   Modes      "2560x1440R" "1600x1200" "1280x1024"
 EndSubSection
EndSection

Section "Extensions"
 Option         "Composite" "Disable"
EndSection
What are those Load lines in the Module section?

We install a piece of software the HP is actively developing called Remote Graphics Software (RGS), it gives you a graphically high quality remote session to a workstation, on display 0 (the X session).  It is a cross platform product (Linux and Windows, no OS X) where the guys at work use it to continue their work when they are remote to the office.  I also use it to connect to my workstation and have the same session (terminals, browser tabs, tools) open and it’s my workstation, giving the impression of native linux when running in full screen mode.  It’s really a very cool tool!

Sources

 

Edit \ Amend Active Directory linux computer objects

At work our Linux machines use AD for authentication, and other things.   Someone decided that they wanted some information in the computer objects description attribute for all computer objects, windows and linux.  Some perl code later and we have a utility that will update the linux computer object’s description attribute with the logged in user, computer model and serial number for all the machines in the orginisation, globally.

Here’s the utility (sanitised): https://github.com/sebastientorres/SysAdmin/blob/master/ADeditAttr.pl

Some things to note:

  • AD is interfaced with using BeyondTrust’s Power Broker Identity Services
  • Perl’s Net LDAP module is used to edit the computer object

If PBIS wasn’t used to interface with AD, I suspect that it would still be possible to do this using another method to find out what the object’s DN is.  Either way, this is how I’ve I done it.  It works and it saves someone spending “a lot of time” manually updating computer objects.

The end result:
Computer objects are beautifully kept up to date, with standard formatting and the removal of human error in performing what is a fairly mundane and repetitive task.

P.S. – Yes, I did write globally, this means about 1000 workstations and servers.  These are managed using CFEngine (v2.10), which copies the script and periodically runs it.