Colplot is a plotting tool designed to work with plot files generated by collectl. It is not a general purpose plotting package.
The main focus for development and testing for colplot has been linux systems running a web server and/or an X server. It has also been tested on PCs and seems to work fine there are well.
There are 4 types of plots colplot is capable of displaying. Rather than increasing the number of different plots by 4-fold, it was decided to select a type for each one that produced the most favorable look for each under most conditions.
As it turns out, some plots look best when displayed one way and can be very difficult to see when displayed another. Perhaps the best example is network data. When communications is in one direction, the use of lines help give one a better feel for the load since the plot is filled with a single color. However, when a system is transmitting and receiving at very close speeds, the second line drawn usually overwrites the first making the first invisible. One solution is a point plot which has no lines and can greatly help show what is really happening. At the same time, if one chooses to zoom in by choosing a narrower time frame, there may be too few points to get a good feel for the data.
When one looks at memory utilization, one may want to see how much of each type is being used and so a line plot show just that. Unlike network data, different types of memory virtually never change at the same rate in the same direction and so data occlusion is rarely a problem. One can easily read values off the left scale and get useful information. However, other times one might be interested in total memory used, making this format more difficult to interpret. A stacked plot literally stacks data points on top of each other so each data point actually represents a total of all those below it, the topmost therefore showing total memory used.
There are many other examples where one format might be more useful than the other so the real solution here it whenever you find yourself having a hard time understanding what you're seeing, consider a different type.
To get started, you first need to make sure colplot has been installed correctly. If you instaled colplot on a linux system running an apache web server via the collectl-utils rpm, all you should need to do is restart apache and you're ready to begin plotting. If not, see the INSTALL document that comes with the kit.
Next you need one or more plottable files. Typically plot files are generated by playing back a collectl raw or raw.gz file using a command like this: "collectl -p filespec -P -f destination-directory -ocz". Alternatively you can use the genplotfiles utility, which can be pretty handy for generating plots files from a set of raw files, especially when used as a cron job. See man collectl for more help and options when running collectl directly or man genplotfiles for the use of that utility.
There are also some sample plot files included in /usr/share/collectl/plotfiles directory which you can easily set by clicking on the ChangeDir> button at the top of the display if you've installed colplot on a Linux machine. To keep the size down, these files only contain a small set of data and you will need to zoom in by picking the appropriate from and thru times to get a good look at the quality of plots colplot is capable of producing
To actually generate a plot, make sure colplot is pointing to the directory in which the plottable files are stored. This can be identified by the second line of the web page and furthermore. Colplot won't let you change to a directory that doesn't contain at least one plottable file in it. Now you simply select one or more plots and hit return or click on the Generate Plot button.
The first step is to make sure your configuration supports it. This also assume you have all the necessary supporting modules such as gnuplot and ghostscript if you want to generate files. If you're running on a Linux machine, you also need to make sure you're running X to display the graphics. If you're running on a PC connected to a Linux machine through a remote terminal session not only does that Linux machine need to be running X you also need to set the display back to your PC on which you have an X Server running. If colplot is installed on a PC, you're ready to begin.
The first thing to do is to make sure colplot is installed correctly and to do this you can just run the command colplot -help to make sure it's there. If you haven't installed it via the rpm the soft-link colplot which points to the executable colplot.pl will not be there and you may need to invoke it in a system-specific manner. At the very top of the display it will show its defaults including the directory it's looking in for plot files.
To generate your first plot simply type colplot.pl -plot cpu and you should see a CPU plot for every file in your default directory. To see what other plot names are available to you type colplot.pl -showplots. The first column is the name of the plot, the second the collectl subsystem it corresponds to and the third column is a description of that plot. See colplot -help for more details.
For more information on genplotfiles.pl type man genplotfiles.
At the most basic level, you create your plot definition in the
allplots section by giving the entry a name, title and list the names
of the columns you wish to plot. If you want to limit the
range of the y-axis, you can set the mask to 1 and include values
for ymin/ymax, noting the default for ymin is 0. You should also include a
descriptive text string for you plot in the
At this point you are essentially done, but you can further customize your plot definitions at the individual field level. In some cases you may want to use a name other than the column header, apply specific ymin/ymax values (which can result in 2 or more y-axes, though colplot limits you to 2) or even divide the plottable values by a constant for the purpose of converting to some other units such as MBs.
Finally, if you want to create your own macro simply give it a name and list the corresponding plot names in the macros section as either a space or comma separate list. Since there are no optional fields involved you do not put these entries in braces.
This is almost certainly related to conflicting data formats in the data file and the plot definition in the .defs file or the column names to plot are just wrong. Carefully example the header of the data file and make sure the column names and data ranges are consistent. If you're still not sure what's going on, consider trying the --showparams switch in CLI mode.
If you installed colplot on a Linux system with RPM, these definitions can be found in /opt/hp/collectl/lib/colplotlib.defs. You can pick up a number of ideas just from examining the parameters used to define the standard plots. It is also possible that some questions you have about colplot may have been answered in the collectl man pages, FAQs or documentation on sourceforge so be sure to check there too.
One often has a variety of files for which data has been collected for different subsystems. These may be from different physical computer systems or simply collected on different days with different settings for collectl. It has been found to be more convenient to silently ignore files for which there is no data to plot. If you think a plot should have been generated, inspect the header of the data file in question and you will usually find it doesn't contain the data you thought it did. Alternatively you may have simply requested the wrong plot. Another typical scenario is requesting detail plots for which there are no detail files.
To make the change permanent set PlotDir= in colplot.conf or append ?dir=dirname to the URL when invoking it.
As it turns out, when you request summary data to be collected, collectl actually has to generate data at the detail level so that it can summarize it. So, for example, if you collect data using collectl -sdn ... you can generate your plot files using collectl -sdnDN ... and get both summary as well as detail data for disks and networks.
NOTE - if you try to playback data that you haven't recorded you will get 0s.
This message is related to lustre data, which uses the -O switch in collectl to specify additional data such as rpc-buffers, metadata, readahead or disk block I/O counts. If one requests a plot of one of these and that data doesn't exist in the file, this message will be produced. This is an exception to the earlier comment about silently ignoring plots because these are rarer and so harder to diagnose without the message when they occur.
As they say, be careful what you ask for because colplot will happily try to do what you ask it to. Most of the time this happens with lustre rpc-buffers or disk block I/O statistics which occur at the OST level and can therefore be a lot of them, especially when there can be multiple plots generated for a single request (such as with the lusblkR macro). This can also happen on systems with many disks, cpus or even networks.
If you really are interested in detail data consider reducing the number of devices by filters or customer plot defintions that specify let information.
This could happen if you are running a vesion of gnuplot that doesn't have the appropriate graphics packages linked in. In order to run from a brower you need png support and from a terminal X11 support. The easiest way to tell is run gnuplot from a terminal window and it should say something like Terminal type set to 'X11'. If not it doesn't support X graphics and will not work from the command line. Now while in gnuplot enter the command set terminal png and gnuplot should respond with Terminal type set to 'png' and if not it won't support web-based graphics. See the next section for even more details.
As discussed in the previous section, colplot requires png or X11 support to run from a browser or command line respectively. As of version 4.1.0, if you try to run colplot from an environment that doesn't support that capability it will not run and you will get the associted error message. Additionally, the -version information displayed has been extended to explicitly state the types of graphics supported by gnuplot as addtional confirmation.
To proceed you will need to install a version of gnuplot that has been built to support these types of graphics OR build it yourself! If building it yourself note that you will need to install the png-devel package to support png graphics which in turn also requires gd and /gd-devel. X11 graphics will require the installation of xorg-x11-devel as well as supporting packages freetype2-devel and fontconfig-devel.
CPU plots are defined to have a Y-Axis range from 0 to 100 and by definition a logarithmic axis must not begin at 0. If you must do this try defining a custom plot that starts at 1.
Some of the CPU plots have been enhanced to add additional data fields that were not captured with eariler versions of collectl. Since these represent real CPU time they should have been included earlier and weren't. Now, cpu time for IRQs, Soft Interrupts and Steal are all included as SysAll as well as SysMore in case you want to be able to differentiate between pure system time and these other fields. If you want to see the breakout by these specific fields, you can choose cpumore in the plot by name field.
If you are getting this error you have two choices - regenerated the plot files with a newer version of collectl and thse new fields will be included. Otherwise you will not be able to select the standard cpu plot and instead will need to choose cpuold by name and keep in mind this will not account for any cpu usage in these 3 unreported fields.