You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 20:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sat Jul 11, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179532262192,59sleep00-21swapper/007:05:060
195762211160,24sleep10-21swapper/107:05:351
196852204168,24sleep30-21swapper/307:07:023
196172203168,24sleep20-21swapper/207:06:092
3087921310,4sleep339-21rcuc/312:25:193
341121100,5sleep01998999cyclictest08:55:240
20004998582,2cyclictest0-21swapper/212:27:202
20004998581,3cyclictest0-21swapper/211:50:482
103582530,4sleep123-21ksoftirqd/111:40:141
254732520,1sleep10-21swapper/112:15:011
137602510,2sleep2311ktimersoftd/208:05:202
3422460,1sleep30-21swapper/307:35:203
186082460,1sleep20-21swapper/210:45:132
192132450,2sleep20-21swapper/208:20:012
20010993822,6cyclictest0-21swapper/309:37:373
1998999358,13cyclictest9952-21munin-run11:40:010
1998999358,12cyclictest11545-21perf10:30:020
2001099340,20cyclictest0-21swapper/309:11:503
1999999349,3cyclictest0-21swapper/109:52:401
1999999341,9cyclictest0-21swapper/111:14:131
20004993329,3cyclictest0-21swapper/211:47:002
1998999336,12cyclictest28511-21idleruntime-cro09:55:010
1998999334,13cyclictest23490-21munin-run08:30:020
20010993219,3cyclictest32183-21smartctl10:00:253
2001099320,20cyclictest0-21swapper/309:58:233
1998999323,14cyclictest8090-21munin-run07:55:010
2001099311,11cyclictest0-21swapper/312:00:213
2001099310,16cyclictest0-21swapper/310:26:383
1999999311,11cyclictest0-21swapper/109:56:321
1998999315,13cyclictest5560-21munin-run11:30:010
1998999312,14cyclictest17502-21munin-run09:30:010
2001099301,15cyclictest0-21swapper/311:42:073
2001099301,15cyclictest0-21swapper/307:44:113
2001099301,14cyclictest0-21swapper/309:17:503
2001099300,12cyclictest0-21swapper/309:30:193
2001099291,12cyclictest0-21swapper/310:39:263
2001099290,15cyclictest0-21swapper/311:52:273
19999992925,3cyclictest0-21swapper/109:04:171
1998999297,10cyclictest0-21swapper/009:10:000
1998999296,11cyclictest5904-21perf07:50:010
1998999290,14cyclictest15935-21perf10:40:020
2001099281,21cyclictest0-21swapper/312:05:273
2001099281,15cyclictest0-21swapper/309:45:173
20010992811,2cyclictest0-21swapper/308:40:123
1999999281,18cyclictest972-21unixbench-2d07:35:261
1999999281,11cyclictest0-21swapper/109:47:201
1998999281,14cyclictest3972-21munin-run12:40:010
2001099271,20cyclictest0-21swapper/312:24:233
2001099271,14cyclictest0-21swapper/310:51:103
2001099271,12cyclictest12091-21df10:30:143
1998999270,12cyclictest0-21swapper/008:55:010
2001099261,18cyclictest0-21swapper/312:15:233
2001099261,17cyclictest0-21swapper/312:12:433
2001099261,13cyclictest22869-21if_enp1s009:40:193
2001099261,13cyclictest0-21swapper/312:30:383
2001099261,13cyclictest0-21swapper/311:18:383
1999999261,17cyclictest21275-21ntp_states07:10:221
1998999263,11cyclictest30710-21idleruntime-cro10:00:000
1998999261,12cyclictest29743-21munin-run12:25:010
2001099258,3cyclictest0-21swapper/311:04:323
2001099251,11cyclictest0-21swapper/307:15:123
2001099251,11cyclictest0-21swapper/307:11:323
20010992510,3cyclictest30302-21/usr/sbin/munin08:45:293
19999992514,2cyclictest8710-21diskmemload12:37:231
1999999251,21cyclictest25912-21missed_timers11:00:201
1999999251,21cyclictest22221-21ntpq12:05:221
1999999251,14cyclictest17344-21perl11:55:161
19999992510,3cyclictest5185-21smart_sda07:45:261
1999999250,16cyclictest0-21swapper/111:47:411
325002240,2sleep20-21swapper/208:50:112
2001099249,3cyclictest0-21swapper/310:55:423
2001099245,2cyclictest0-21swapper/308:35:303
2001099241,20cyclictest0-21swapper/308:33:103
2001099241,19cyclictest0-21swapper/310:45:503
2001099241,13cyclictest0-21swapper/311:10:463
2001099241,13cyclictest0-21swapper/311:09:523
2001099241,12cyclictest0-21swapper/310:09:233
2001099240,14cyclictest0-21swapper/309:21:473
20004992410,3cyclictest9377-21ntpq07:55:222
1999999247,4cyclictest20957-21munin-run12:05:011
1999999241,9cyclictest0-21swapper/109:39:121
1999999241,3cyclictest16793-21gltestperf10:40:171
19999992413,2cyclictest0-21swapper/110:18:441
1999999241,20cyclictest24433-21threads07:15:281
1999999241,20cyclictest23811-21ntp_states10:55:231
1999999241,15cyclictest0-21swapper/110:29:291
1999999241,14cyclictest0-21swapper/111:21:071
1999999241,11cyclictest8710-21diskmemload09:15:021
1999999241,11cyclictest7703-21diskstats10:20:121
1999999241,11cyclictest7703-21diskstats10:20:121
1999999241,11cyclictest579-21systemd-journal10:40:011
1999999241,11cyclictest21281-21latency_hist08:25:011
1999999241,11cyclictest11977-21expr10:30:131
2001099231,5cyclictest0-21swapper/310:18:233
2001099231,15cyclictest0-21swapper/310:44:503
1999999238,3cyclictest29897-21cat12:25:011
1999999238,3cyclictest13086-21cat09:19:561
1999999233,8cyclictest0-21swapper/111:25:161
19999992313,2cyclictest0-21swapper/110:13:581
1999999231,19cyclictest827-21runrttasks11:52:451
1999999231,14cyclictest0-21swapper/111:05:081
19999992311,2cyclictest827-21runrttasks10:50:121
1999999231,12cyclictest0-21swapper/107:20:241
1999999231,11cyclictest27705-21cat12:20:011
1999999231,11cyclictest17877-21cat08:15:171
1999999231,10cyclictest18956-21gunzip09:30:241
1998999237,6cyclictest0-21swapper/009:37:430
1998999232,8cyclictest24632-21kworker/0:311:08:520
1998999232,8cyclictest14131-21kworker/0:012:07:160
1998999231,19cyclictest24734-21awk11:00:010
1998999231,11cyclictest13119-21perf09:20:010
2001099223,4cyclictest14693-21munin-run08:10:013
2001099222,3cyclictest0-21swapper/307:25:533
2001099221,13cyclictest0-21swapper/311:49:033
2001099221,13cyclictest0-21swapper/308:56:323
2001099221,13cyclictest0-21swapper/307:45:313
2000499221,6cyclictest0-21swapper/210:09:212
20004992211,3cyclictest827-21runrttasks10:35:482
1999999228,2cyclictest0-21swapper/110:50:021
1999999226,2cyclictest16589-21ntpq09:25:221
1999999226,2cyclictest0-21swapper/111:30:161
19999992212,2cyclictest0-21swapper/111:15:251
1999999221,20cyclictest0-21swapper/110:03:461
1999999221,20cyclictest0-21swapper/108:04:511
1999999221,20cyclictest0-21swapper/108:04:511
1999999221,20cyclictest0-21swapper/107:34:121
1999999221,19cyclictest0-21swapper/108:53:321
1999999221,19cyclictest0-21swapper/108:45:201
1999999221,19cyclictest0-21swapper/108:38:421
1999999221,18cyclictest7974-21chrt11:35:081
1999999221,10cyclictest29988-21grep12:25:121
1998999223,16cyclictest25064-21munin-run07:20:010
19989992217,3cyclictest26362-21fschecks_count08:35:150
2001099211,6cyclictest0-21swapper/310:20:203
2001099211,6cyclictest0-21swapper/310:20:193
2001099211,3cyclictest5207-21hddtemp_smartct09:00:163
2001099211,17cyclictest0-21swapper/308:50:223
2001099211,13cyclictest0-21swapper/311:36:303
2001099211,13cyclictest0-21swapper/311:25:343
2001099211,11cyclictest0-21swapper/309:07:383
2001099210,14cyclictest0-21swapper/312:35:103
2000499211,6cyclictest0-21swapper/210:34:032
2000499211,6cyclictest0-21swapper/209:21:562
2000499211,4cyclictest0-21swapper/212:11:012
1999999218,2cyclictest1036-21/usr/sbin/munin12:30:251
1999999211,9cyclictest0-21swapper/108:40:171
1999999211,8cyclictest0-21swapper/109:44:591
19999992112,2cyclictest0-21swapper/110:08:181
1999999211,14cyclictest0-21swapper/109:22:211
1998999218,3cyclictest31391-21fschecks_time10:00:150
1998999217,6cyclictest0-21swapper/011:58:380
1998999215,6cyclictest0-21swapper/011:50:150
1998999215,2cyclictest3031-21kworker/0:011:23:320
19989992110,3cyclictest16147-21/usr/sbin/munin10:40:250
2001099205,4cyclictest3381-21tune2fs10:10:173
2001099203,14cyclictest17419-21gltestperf11:55:173
2001099202,8cyclictest87350irq/126-enp1s0-08:20:333
2001099201,6cyclictest25627-21df07:20:143
20010992013,4cyclictest1-21systemd08:10:203
2001099201,12cyclictest0-21swapper/309:53:033
2001099201,11cyclictest5785-21/usr/sbin/munin11:30:253
2001099200,3cyclictest87550irq/128-enp1s0-08:03:333
2001099200,3cyclictest87550irq/128-enp1s0-08:03:323
2001099200,11cyclictest401ktimersoftd/307:50:173
2000499207,3cyclictest0-21swapper/210:19:022
2000499201,6cyclictest0-21swapper/210:54:272
2000499201,6cyclictest0-21swapper/209:37:262
2000499201,6cyclictest0-21swapper/209:31:322
2000499201,5cyclictest0-21swapper/210:58:432
20004992014,3cyclictest16775-21gltestperf10:40:162
20004992013,4cyclictest24201-21/usr/sbin/munin07:15:252
1999999208,3cyclictest0-21swapper/107:29:171
1999999203,2cyclictest24522-21kworker/1:207:54:051
1999999201,9cyclictest3312-21users07:40:261
1999999201,4cyclictest579-21systemd-journal09:10:011
1999999201,2cyclictest0-21swapper/108:25:231
1999999201,16cyclictest24624-21memory08:30:181
1999999200,18cyclictest0-21swapper/107:59:521
1999999200,10cyclictest0-21swapper/108:55:231
1998999209,8cyclictest30904-21missed_timers12:25:200
1998999209,3cyclictest25289-21/usr/sbin/munin07:20:230
1998999206,6cyclictest0-21swapper/010:34:300
1998999203,10cyclictest10179-21/usr/sbin/munin11:40:200
1998999201,11cyclictest8315-21/usr/sbin/munin07:55:170
1998999201,11cyclictest2968-21/usr/sbin/munin10:10:190
1998999200,8cyclictest0-21swapper/012:12:130
223122191,3sleep00-21swapper/010:52:310
2001099194,7cyclictest87450irq/127-enp1s0-08:15:243
2001099194,2cyclictest87550irq/128-enp1s0-07:58:173
2001099191,5cyclictest30158-21fschecks_time07:30:173
2001099191,10cyclictest1361-21/usr/sbin/munin11:20:233
2000499191,7cyclictest0-21swapper/209:19:512
2000499191,4cyclictest0-21swapper/210:23:072
2000499191,4cyclictest0-21swapper/210:23:072
20004991914,3cyclictest30157-21irqstats11:10:172
1999999191,2cyclictest0-21swapper/108:13:091
1998999194,4cyclictest0-21swapper/010:17:380
1998999193,9cyclictest0-21swapper/011:18:260
1998999191,10cyclictest13329-21/usr/sbin/munin09:20:190
1998999191,10cyclictest0-21swapper/009:00:350
1998999190,9cyclictest0-21swapper/011:12:410
92012181,2sleep00-21swapper/010:23:300
92012181,2sleep00-21swapper/010:23:300
2001099186,7cyclictest0-21swapper/308:26:123
2001099181,5cyclictest15763-21expr09:25:143
2000499186,3cyclictest17051-21unixbench_multi09:25:282
2000499184,7cyclictest755-21rngd12:15:032
2000499182,10cyclictest0-21swapper/211:08:372
2000499181,7cyclictest32279-21perf08:50:002
2000499181,7cyclictest0-21swapper/207:25:262
2000499181,6cyclictest0-21swapper/210:01:022
2000499181,4cyclictest0-21swapper/211:43:402
2000499181,4cyclictest0-21swapper/209:40:422
2000499181,3cyclictest87350irq/126-enp1s0-12:00:192
2000499181,3cyclictest27266-21munin-run07:25:022
20004991813,3cyclictest23710-21/usr/sbin/munin08:30:232
2000499181,11cyclictest0-21swapper/212:35:302
2000499181,10cyclictest0-21swapper/212:21:172
19999991814,3cyclictest0-21swapper/108:07:031
1998999189,2cyclictest0-21swapper/009:40:230
1998999186,3cyclictest0-21swapper/009:32:050
1998999186,2cyclictest28274-21kworker/0:210:06:440
1998999185,4cyclictest0-21swapper/007:44:550
1998999185,11cyclictest29047-21head07:25:250
1998999184,8cyclictest0-21swapper/011:31:230
1998999181,4cyclictest9719-21if_err_enp1s009:10:180
1998999181,11cyclictest0-21swapper/008:45:160
1998999181,10cyclictest0-21swapper/010:45:250
1998999180,8cyclictest0-21swapper/011:46:200
261932171,2sleep00-21swapper/009:48:490
2000499175,3cyclictest87350irq/126-enp1s0-09:10:202
2000499174,4cyclictest579-21systemd-journal11:56:172
2000499173,3cyclictest6342-21fschecks_time11:30:172
2000499172,7cyclictest1722-21/usr/sbin/munin07:40:242
2000499171,7cyclictest20573-21sensors_temp08:20:232
2000499171,6cyclictest31902-21grep07:35:122
2000499171,5cyclictest0-21swapper/210:25:482
20004991713,3cyclictest0-21swapper/211:27:082
20004991713,3cyclictest0-21swapper/207:45:522
20004991713,3cyclictest0-21swapper/207:30:032
20004991712,3cyclictest9580-21vmstat11:35:282
20004991712,3cyclictest20623-21grep07:10:152
2000499171,14cyclictest0-21swapper/211:20:112
2000499171,11cyclictest0-21swapper/210:14:102
2000499171,10cyclictest0-21swapper/212:30:162
1998999175,3cyclictest0-21swapper/012:18:140
1998999173,9cyclictest0-21swapper/007:35:210
1998999172,6cyclictest30365-21hddtemp_smartct07:30:170
1998999172,12cyclictest12718-21/usr/sbin/munin08:05:130
1998999171,6cyclictest365-21sed12:30:180
1998999171,14cyclictest0-21swapper/007:11:150
1998999171,10cyclictest0-21swapper/012:00:140
1998999171,10cyclictest0-21swapper/011:01:250
8242161,2sleep20-21swapper/211:16:292
2000499166,3cyclictest87650irq/129-enp1s0-11:00:122
2000499166,3cyclictest87350irq/126-enp1s0-08:25:112
2000499164,3cyclictest30331-21sshd09:55:312
2000499163,7cyclictest0-21swapper/209:50:252
2000499163,2cyclictest87550irq/128-enp1s0-08:35:112
2000499161,6cyclictest0-21swapper/209:05:202
2000499161,14cyclictest0-21swapper/208:40:532
20004991611,3cyclictest6116-21/usr/sbin/munin07:50:202
20004991611,3cyclictest3046-21idleruntime08:55:182
20004991611,3cyclictest25403-21open_files09:45:242
20004991611,3cyclictest12053-21unixbench_singl08:00:252
20004991611,3cyclictest12053-21unixbench_singl08:00:252
1998999161,6cyclictest0-21swapper/008:40:170
19989991611,3cyclictest20109-21cat08:20:180
19989991611,3cyclictest18179-21ntp_states08:15:220
19989991611,3cyclictest10702-21expr08:00:120
19989991611,3cyclictest10702-21expr08:00:120
19989991610,4cyclictest0-21swapper/008:30:170
2000499159,3cyclictest5430-21cat09:00:182
2000499151,3cyclictest0-21swapper/208:10:062
20004991510,3cyclictest21181-21/usr/sbin/munin12:05:142
1998999159,3cyclictest15008-21cpuspeed_turbos08:10:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional