You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 15:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Aug 12, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
174892271193,20sleep00-21swapper/007:08:130
174012203170,22sleep10-21swapper/107:07:061
173982203166,24sleep30-21swapper/307:07:033
173452202165,24sleep20-21swapper/207:06:222
1212221560,5sleep01770899cyclictest10:35:120
17720999896,1cyclictest0-21swapper/209:25:362
90492610,2sleep10-21swapper/108:00:191
325562520,2sleep01770899cyclictest08:55:120
48412500,1sleep10-21swapper/109:05:111
1772699324,16cyclictest20382-21cut09:40:133
1772699302,17cyclictest20351-21crond10:55:013
1771699301,15cyclictest0-21swapper/110:52:491
1772099297,10cyclictest29072-21switchtime12:25:242
1771699299,10cyclictest17321-21expr08:20:141
1771699299,10cyclictest17321-21expr08:20:141
17716992913,3cyclictest18717-21cat09:35:201
1772699288,12cyclictest8979-21/usr/sbin/munin09:15:153
1772699286,15cyclictest808-21snmpd10:42:053
1772699286,15cyclictest808-21snmpd10:42:053
1772699281,12cyclictest25622-21gltestperf11:05:153
1772099282,9cyclictest0-21swapper/210:30:172
1770899289,16cyclictest7168-21munin-run10:25:000
1770899281,6cyclictest0-21swapper/012:25:230
1770899281,6cyclictest0-21swapper/009:12:200
1770899281,12cyclictest15930-40dnf11:55:230
1770899281,12cyclictest15930-40dnf11:55:220
1772699278,9cyclictest29993-21find10:00:243
1772699276,12cyclictest17530-21latency_hist09:35:023
1772699276,12cyclictest17530-21latency_hist09:35:023
1772699272,13cyclictest1397-21/usr/sbin/munin11:25:173
17716992722,2cyclictest1312-21unixbench_multi12:35:251
1771699270,21cyclictest0-21swapper/110:40:281
1771699270,21cyclictest0-21swapper/110:40:281
1770899279,4cyclictest0-21swapper/011:45:160
1770899271,6cyclictest0-21swapper/009:23:100
1770899270,6cyclictest0-21swapper/010:28:160
1772699267,12cyclictest24406-21cpu09:50:113
1772699265,9cyclictest31554-21/usr/sbin/munin08:50:243
1772699264,15cyclictest22989-1kworker/3:0H07:10:263
1772699262,15cyclictest9186-21processes11:40:223
1772699262,12cyclictest25204-21/usr/sbin/munin07:25:253
17726992610,7cyclictest7075-21smart_sda11:35:243
17716992620,3cyclictest26247-21diskstats08:40:161
1771699261,21cyclictest482-21perf10:10:021
1771699261,20cyclictest0-21swapper/109:52:331
1770899261,11cyclictest1059-21sshd11:23:460
17708992610,5cyclictest30809-21kernelversion12:30:190
1772699256,6cyclictest11470-21kworker/u8:011:30:183
1772699256,16cyclictest21505-21/usr/sbin/munin08:30:223
1772699255,12cyclictest31908-21cstates11:20:123
1772699253,6cyclictest0-21swapper/312:03:413
1772699252,5cyclictest0-21swapper/312:30:233
1772699252,13cyclictest6548-21diskmemload09:10:173
1772699251,6cyclictest0-21swapper/311:45:313
1772699251,11cyclictest29282-21sed12:25:253
1772699250,10cyclictest0-21swapper/309:35:103
1772099251,5cyclictest0-21swapper/210:36:522
1772099251,5cyclictest0-21swapper/210:10:082
17716992513,3cyclictest5200-21users11:30:261
1771699251,12cyclictest0-21swapper/110:15:201
1771699251,11cyclictest12551-21cpuspeed_turbos11:50:141
1771699250,23cyclictest0-21swapper/109:20:001
1770899252,11cyclictest22162-21/usr/sbin/munin09:45:170
1770899251,5cyclictest0-21swapper/010:49:510
1770899251,11cyclictest23405-21/usr/sbin/munin12:15:260
1772699247,8cyclictest9586-21/usr/sbin/munin10:30:133
1772699245,8cyclictest32191-21/usr/sbin/munin12:35:143
1772699245,7cyclictest6548-21diskmemload11:55:113
1772699245,7cyclictest6548-21diskmemload11:55:103
1772699245,3cyclictest26869-21swap12:20:263
1772699245,11cyclictest29465-21cpuspeed_turbos11:15:123
1772699243,9cyclictest958-21cstates10:10:113
1772699243,7cyclictest14657-21latency_hist08:15:013
1772699242,8cyclictest0-21swapper/310:15:133
1772699241,14cyclictest13137-21latency_hist09:25:023
1772699241,10cyclictest0-21swapper/310:45:263
1772699240,17cyclictest0-21swapper/310:20:153
1772099248,8cyclictest8564-1kworker/u9:112:05:442
1772099246,9cyclictest6548-21diskmemload11:45:182
1772099242,7cyclictest0-21swapper/210:42:022
1772099242,7cyclictest0-21swapper/210:42:022
1772099241,8cyclictest0-21swapper/209:49:382
1772099241,6cyclictest0-21swapper/212:12:242
1772099241,6cyclictest0-21swapper/211:28:062
1772099241,6cyclictest0-21swapper/211:24:012
1772099241,6cyclictest0-21swapper/209:19:262
1772099241,5cyclictest0-21swapper/212:22:012
1772099241,4cyclictest0-21swapper/212:35:122
1772099240,11cyclictest0-21swapper/210:05:342
1771699249,3cyclictest18591-21hddtemp_smartct07:10:171
1771699243,4cyclictest0-21swapper/110:20:271
17716992420,3cyclictest0-21swapper/109:56:211
17716992418,3cyclictest20155-21expr09:40:121
1771699241,18cyclictest0-21swapper/111:27:061
1771699241,18cyclictest0-21swapper/109:48:241
1771699241,10cyclictest5935-21chrtset11:35:141
17716992410,9cyclictest0-21swapper/111:10:231
1771699240,19cyclictest0-21swapper/109:12:051
1771699240,18cyclictest0-21swapper/112:34:341
1770899241,5cyclictest0-21swapper/012:05:430
1770899241,5cyclictest0-21swapper/011:38:200
1770899241,5cyclictest0-21swapper/010:57:160
1770899241,3cyclictest2998-21/usr/sbin/munin10:15:160
1770899241,10cyclictest827-21runrttasks11:25:500
1770899241,10cyclictest28302-21taskset09:57:180
1770899240,3cyclictest9321-21kworker/0:110:32:440
241852230,1sleep30-21swapper/307:20:243
222732231,18sleep122333-21sed08:30:181
1772699238,7cyclictest22846-21awk11:00:143
1772699238,6cyclictest21151-21df_inode07:15:133
1772699238,5cyclictest20349-21ntp_kernel_pll_08:25:223
1772699235,7cyclictest27980-21timerwakeupswit09:55:243
1772699235,10cyclictest31813-21/usr/sbin/munin07:40:233
1772699234,7cyclictest8186-21/usr/sbin/munin10:25:193
1772699233,8cyclictest31089-21cpuspeed_turbos10:05:133
1772699233,8cyclictest29817-21expr07:35:113
1772699232,9cyclictest28543-21fschecks_count08:45:153
1772699232,9cyclictest0-21swapper/309:00:193
1772699232,12cyclictest6772-21kworker/u8:209:45:233
17726992315,5cyclictest12751-21diskstats11:50:153
17726992311,5cyclictest21553-21df_inode12:10:153
17726992310,4cyclictest11888-21cut10:35:123
1772699230,13cyclictest19079-21latency_hist08:25:023
1772699230,13cyclictest19079-21latency_hist08:25:023
1772099236,5cyclictest32642-21tr07:40:182
1772099231,6cyclictest0-21swapper/209:20:362
1772099231,5cyclictest0-21swapper/212:04:432
1772099231,5cyclictest0-21swapper/211:50:292
1772099231,4cyclictest0-21swapper/209:36:272
1772099231,16cyclictest0-21swapper/209:50:352
17720992310,3cyclictest4457-21unixbench-2d10:15:262
1771699239,3cyclictest32389-21irqstats11:20:181
1771699236,3cyclictest30575-21systemd-journal10:03:071
17716992320,2cyclictest0-21swapper/110:37:541
17716992313,3cyclictest31590-21munin-run07:40:021
1771699231,20cyclictest0-21swapper/112:00:231
1771699231,19cyclictest23120-21df11:00:141
1771699231,19cyclictest0-21swapper/110:45:161
1771699231,11cyclictest16885-21munin-run08:20:011
17716992310,3cyclictest20574-21sh08:25:261
1771699230,3cyclictest0-21swapper/112:15:081
1771699230,17cyclictest0-21swapper/112:12:031
1771699230,16cyclictest0-21swapper/112:20:401
1770899231,5cyclictest0-21swapper/012:11:180
1770899231,3cyclictest0-21swapper/009:42:290
1770899231,12cyclictest7663-21kworker/u8:111:40:140
298022221,3sleep029826-21ntp_states10:00:230
1772699229,3cyclictest22989-1kworker/3:0H07:45:143
1772699223,6cyclictest7663-21kworker/u8:112:07:153
1772699223,6cyclictest12050irq/123-ahci[0008:00:003
1772699222,9cyclictest20584-21/usr/sbin/munin10:55:173
1772699220,13cyclictest0-21swapper/308:40:023
1772099227,8cyclictest8564-1kworker/u9:110:20:472
1772099226,8cyclictest87650irq/129-enp1s0-11:40:202
1772099221,6cyclictest0-21swapper/212:15:082
1772099221,6cyclictest0-21swapper/211:13:232
1772099221,15cyclictest0-21swapper/208:28:442
1772099221,13cyclictest0-21swapper/211:17:472
1772099221,11cyclictest22843-21threads08:30:242
1772099221,11cyclictest0-21swapper/209:41:072
1772099221,10cyclictest0-21swapper/211:00:482
1772099221,10cyclictest0-21swapper/210:27:582
1772099220,16cyclictest0-21swapper/209:07:152
1772099220,12cyclictest0-21swapper/210:53:182
1771699229,3cyclictest0-21swapper/108:49:521
1771699225,2cyclictest88950irq/132-enp2s0-10:30:561
1771699222,12cyclictest0-21swapper/108:09:491
17716992219,2cyclictest0-21swapper/109:32:461
17716992219,2cyclictest0-21swapper/109:32:461
17716992217,3cyclictest0-21swapper/107:53:111
1771699221,19cyclictest11470-21kworker/u8:011:45:071
1771699221,18cyclictest1418-21gltestperf10:10:151
1771699221,10cyclictest28691-21sed12:25:181
1771699220,20cyclictest0-21swapper/109:25:131
1771699220,17cyclictest0-21swapper/111:56:351
1771699220,17cyclictest0-21swapper/111:56:351
1771699220,17cyclictest0-21swapper/110:57:531
1770899227,3cyclictest20370-21idleruntime-cro10:55:010
1770899221,9cyclictest0-21swapper/010:41:590
1770899221,9cyclictest0-21swapper/010:41:590
1770899221,5cyclictest0-21swapper/009:36:480
1772699217,6cyclictest0-21swapper/309:26:093
1772699216,3cyclictest29368-21dump-pmu-power07:35:003
1772699213,15cyclictest15508-21gltestperf08:15:163
17726992114,4cyclictest5135-21perl09:05:153
1772099215,4cyclictest6856-21ntpq11:35:212
1772099214,9cyclictest0-21swapper/211:32:452
1772099212,10cyclictest31061-21ntp_states12:30:232
1772099211,7cyclictest10476-21/usr/sbin/munin08:05:132
1772099211,6cyclictest579-21systemd-journal08:55:022
1772099211,5cyclictest0-21swapper/209:59:122
1772099211,11cyclictest6802-21cpu09:10:122
1772099211,11cyclictest5463-21users07:50:272
1772099211,11cyclictest20927-21df_inode10:55:142
1772099211,10cyclictest0-21swapper/209:30:222
1772099211,10cyclictest0-21swapper/209:30:222
1771699216,4cyclictest19363-21diskstats12:05:181
1771699215,8cyclictest25325-21df11:05:121
1771699211,18cyclictest0-21swapper/107:55:181
1771699211,17cyclictest31425-21pmu-power08:50:221
1771699211,14cyclictest0-21swapper/107:21:281
1771699211,11cyclictest12487-21ls09:20:251
1771699210,19cyclictest0-21swapper/108:10:161
1771699210,17cyclictest0-21swapper/110:27:311
1771699210,15cyclictest0-21swapper/111:15:571
1770899217,5cyclictest0-21swapper/011:11:190
1770899211,5cyclictest0-21swapper/011:04:060
1770899211,5cyclictest0-21swapper/010:09:200
1770899210,8cyclictest8564-1kworker/u9:109:25:240
140502211,7sleep00-21swapper/011:51:030
1772699201,4cyclictest0-21swapper/311:13:113
17726992013,5cyclictest4547-21/usr/sbin/munin07:50:163
1772699200,8cyclictest0-21swapper/312:15:153
1772099205,3cyclictest27822-21sort07:30:152
1772099203,4cyclictest25192-21expr11:05:112
1772099202,9cyclictest0-21swapper/210:00:192
1772099202,8cyclictest19882-21crond07:15:012
1772099201,9cyclictest27168-21sh07:30:002
17720992018,1cyclictest0-21swapper/207:15:212
1772099201,6cyclictest14596-21/usr/sbin/munin11:55:132
1772099201,6cyclictest14596-21/usr/sbin/munin11:55:132
1772099201,11cyclictest839-21lldpd08:37:372
1772099201,11cyclictest3526-21sshd07:48:392
1772099201,10cyclictest30061-21sh08:50:022
1772099201,10cyclictest23005-21/usr/sbin/munin07:20:182
1772099201,10cyclictest17487-21grep10:45:252
1771699206,8cyclictest0-21swapper/108:35:211
17716992016,3cyclictest0-21swapper/107:47:111
17716992013,4cyclictest28212-21irqstats07:30:191
17716992012,2cyclictest32540-21if_enp1s007:40:171
1771699201,16cyclictest8345-21df11:40:141
1771699200,18cyclictest0-21swapper/107:30:021
1770899206,3cyclictest0-21swapper/009:52:060
1770899204,5cyclictest0-21swapper/011:05:090
1770899204,14cyclictest4372-21perf09:05:020
1770899204,13cyclictest24979-21munin-run07:25:010
1770899201,7cyclictest0-21swapper/012:00:160
1770899201,4cyclictest23703-21/usr/sbin/munin08:35:160
1770899201,10cyclictest0-21swapper/009:32:010
1770899201,10cyclictest0-21swapper/009:32:010
275412191,3sleep00-21swapper/012:24:450
17726991913,4cyclictest25899-21/usr/sbin/munin08:40:143
17726991912,4cyclictest371-21diskstats08:55:143
17726991910,6cyclictest10476-21/usr/sbin/munin08:05:203
1772099196,3cyclictest87350irq/126-enp1s0-08:55:202
1772099195,3cyclictest87650irq/129-enp1s0-07:55:232
1772099194,3cyclictest3890-21unixbench_singl09:00:252
1772099191,6cyclictest808-21snmpd07:36:282
1771699191,3cyclictest0-21swapper/107:18:001
1770899193,3cyclictest0-21swapper/012:35:560
1770899192,14cyclictest18637-21grep08:20:270
1770899192,14cyclictest18637-21grep08:20:270
1770899191,9cyclictest0-21swapper/007:13:380
1770899191,8cyclictest87350irq/126-enp1s0-11:30:260
1770899191,5cyclictest0-21swapper/009:15:060
1770899191,12cyclictest0-21swapper/010:10:180
17726991810,5cyclictest8271-21/usr/sbin/munin08:00:183
1772099185,3cyclictest87350irq/126-enp1s0-08:40:162
1772099181,9cyclictest0-21swapper/208:20:502
1772099181,9cyclictest0-21swapper/208:20:502
1772099181,5cyclictest8855-21gltestperf08:00:172
1772099181,5cyclictest12668-21/usr/sbin/munin08:10:182
1771699186,2cyclictest2366-21bc09:00:101
1771699181,15cyclictest1170-21ntpq08:55:201
1770899186,3cyclictest12050irq/123-ahci[0008:30:170
1770899181,6cyclictest0-21swapper/007:50:120
1770899181,5cyclictest27403-21/usr/sbin/munin07:30:160
1770899181,10cyclictest0-21swapper/007:55:160
1770899180,6cyclictest0-21swapper/011:15:200
1770899180,5cyclictest0-21swapper/008:05:190
1772099173,3cyclictest87350irq/126-enp1s0-08:15:232
1770899178,6cyclictest16507-21sshd08:15:420
1770899174,2cyclictest87550irq/128-enp1s0-09:05:190
1770899172,12cyclictest13323-21head08:10:150
1770899171,6cyclictest0-21swapper/007:25:260
1770899171,4cyclictest19311-21/usr/sbin/munin08:25:190
1770899171,3cyclictest25899-21/usr/sbin/munin08:40:230
1770899171,15cyclictest0-21swapper/007:39:210
17708991711,3cyclictest31102-21irqstats08:50:170
1770899170,3cyclictest21194-21fschecks_count07:15:140
1770899165,3cyclictest1676-21/usr/sbin/munin07:45:220
1770899162,12cyclictest32469-21head07:40:150
1770899161,5cyclictest0-21swapper/008:01:440
17708991611,3cyclictest29626-21sed08:45:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional