You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon May 11, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
194602232164,56sleep20-21swapper/219:05:322
197502203167,24sleep30-21swapper/319:09:213
195532202166,24sleep10-21swapper/119:06:441
194982201163,25sleep00-21swapper/019:06:040
19901995653,2cyclictest0-21swapper/200:32:292
19901995551,3cyclictest0-21swapper/220:27:332
72352500,3sleep30-21swapper/321:05:173
158522490,2sleep121-21rcuc/123:50:251
1988999382,25cyclictest21156-21kworker/0:219:15:280
1990699337,3cyclictest0-21swapper/322:47:273
1988999321,10cyclictest12974-21kworker/0:221:22:570
1989599314,18cyclictest13662-21sshd23:45:271
1988999312,22cyclictest28173-21kworker/0:221:57:420
1990699304,4cyclictest0-21swapper/322:54:453
1990199307,3cyclictest0-21swapper/223:51:002
1990699298,3cyclictest0-21swapper/323:08:493
1990699296,3cyclictest0-21swapper/322:28:453
1990699296,3cyclictest0-21swapper/322:28:453
1990699291,25cyclictest7279-21/usr/sbin/munin19:50:233
1990199297,3cyclictest0-21swapper/222:09:572
39242280,4sleep011-21rcuc/000:40:010
1990699286,3cyclictest0-21swapper/320:43:393
1990699286,3cyclictest0-21swapper/319:19:163
1990699285,3cyclictest0-21swapper/319:14:593
1990699284,3cyclictest0-21swapper/319:22:473
1990699281,23cyclictest0-21swapper/323:51:063
1990699280,25cyclictest0-21swapper/321:47:233
1990199284,3cyclictest0-21swapper/220:46:322
1988999281,6cyclictest0-21swapper/023:00:020
1988999281,5cyclictest0-21swapper/023:10:220
1988999281,5cyclictest0-21swapper/021:47:230
1988999281,11cyclictest22097-21kworker/0:222:54:300
1990699275,3cyclictest0-21swapper/300:07:043
1990699274,3cyclictest0-21swapper/323:41:023
1990699274,3cyclictest0-21swapper/322:04:583
1990699274,3cyclictest0-21swapper/320:20:513
1990699273,3cyclictest0-21swapper/320:29:033
1990699271,24cyclictest0-21swapper/322:10:073
1990699271,23cyclictest0-21swapper/322:59:163
1990699270,7cyclictest0-21swapper/323:34:313
1990699270,24cyclictest0-21swapper/321:42:553
1990699270,24cyclictest0-21swapper/300:15:343
1990699270,24cyclictest0-21swapper/300:10:233
1990199276,3cyclictest0-21swapper/223:04:522
1990199274,3cyclictest0-21swapper/220:22:432
1990199274,3cyclictest0-21swapper/219:54:592
1990199274,3cyclictest0-21swapper/200:29:072
1990199273,3cyclictest11840-21irqstats21:15:202
1990199273,3cyclictest11840-21irqstats21:15:202
1988999274,11cyclictest29027-21perf23:10:010
1988999271,6cyclictest0-21swapper/000:15:340
1988999271,5cyclictest0-21swapper/022:26:410
1988999271,5cyclictest0-21swapper/022:26:410
1988999271,4cyclictest0-21swapper/000:10:230
310782260,4sleep231076-21grep19:30:262
1990699269,3cyclictest0-21swapper/323:39:223
1990699269,3cyclictest0-21swapper/323:39:223
1990699268,3cyclictest0-21swapper/320:52:003
1990699267,3cyclictest0-21swapper/300:31:463
1990699264,3cyclictest0-21swapper/321:02:593
1990699263,3cyclictest0-21swapper/300:00:163
1990699263,15cyclictest0-21swapper/300:35:043
1990699261,22cyclictest0-21swapper/321:12:203
1990699260,24cyclictest0-21swapper/321:37:503
1990699260,23cyclictest0-21swapper/323:56:533
1990199263,3cyclictest0-21swapper/221:06:122
1989599262,16cyclictest24551-21sshd22:59:271
1989599261,10cyclictest0-21swapper/120:36:381
1988999263,4cyclictest0-21swapper/023:51:050
1988999262,10cyclictest6290-21kworker/0:121:26:010
1988999261,6cyclictest0-21swapper/022:47:440
1988999261,6cyclictest0-21swapper/022:21:190
1988999261,6cyclictest0-21swapper/021:37:500
1988999261,5cyclictest14285-21kworker/u8:321:42:550
1988999261,5cyclictest0-21swapper/022:32:520
1988999261,4cyclictest0-21swapper/021:50:180
1988999261,3cyclictest0-21swapper/019:50:230
19889992610,4cyclictest4420-21/usr/sbin/munin21:00:250
1988999260,8cyclictest27252-21kworker/0:000:21:380
1988999260,11cyclictest32738-21kworker/0:022:10:540
185052260,2sleep121-21rcuc/121:30:191
1990699251,22cyclictest0-21swapper/323:10:043
1990699251,22cyclictest0-21swapper/321:16:473
1990699251,22cyclictest0-21swapper/321:16:473
1990699250,23cyclictest0-21swapper/323:49:373
1990699250,23cyclictest0-21swapper/322:30:053
1990699250,22cyclictest0-21swapper/323:26:583
1990699250,22cyclictest0-21swapper/322:39:583
1990699250,22cyclictest0-21swapper/322:05:433
1990699250,10cyclictest0-21swapper/321:24:423
1990199254,3cyclictest0-21swapper/223:27:582
1990199254,3cyclictest0-21swapper/221:20:402
1990199254,3cyclictest0-21swapper/219:43:152
1990199253,3cyclictest0-21swapper/220:19:412
1990199252,3cyclictest87450irq/127-enp1s0-23:40:582
1989599252,13cyclictest21424-21diskstats00:05:141
1989599251,11cyclictest0-21swapper/122:21:231
1988999251,6cyclictest0-21swapper/023:43:100
1988999251,6cyclictest0-21swapper/000:09:140
1988999251,5cyclictest0-21swapper/023:56:520
1988999251,5cyclictest0-21swapper/023:49:360
1988999251,5cyclictest0-21swapper/021:16:460
1988999251,5cyclictest0-21swapper/021:16:460
1988999251,5cyclictest0-21swapper/000:03:120
1988999251,4cyclictest0-21swapper/021:12:190
19889992511,3cyclictest23388-21awk20:30:020
1988999251,12cyclictest29874-21/usr/sbin/munin00:25:260
1990699241,21cyclictest0-21swapper/322:21:183
1990699241,21cyclictest0-21swapper/300:25:043
1990699241,12cyclictest0-21swapper/319:27:133
1990699240,22cyclictest0-21swapper/322:16:483
1990699240,22cyclictest0-21swapper/300:24:483
1990199247,3cyclictest0-21swapper/223:33:102
1990199243,2cyclictest87650irq/129-enp1s0-23:19:182
1990199243,2cyclictest0-21swapper/219:57:472
1990199242,2cyclictest88950irq/132-enp2s0-22:52:522
19901992415,7cyclictest8760-21latency23:35:202
19901992415,7cyclictest8760-21latency23:35:202
1989599247,4cyclictest0-21swapper/122:48:191
1989599242,15cyclictest0-21swapper/100:21:301
1989599241,7cyclictest0-21swapper/122:27:241
1989599241,7cyclictest0-21swapper/122:27:241
1989599241,6cyclictest0-21swapper/100:30:171
1989599241,13cyclictest0-21swapper/121:22:341
1989599241,10cyclictest0-21swapper/123:35:291
1989599241,10cyclictest0-21swapper/123:35:291
1988999249,3cyclictest2223-21/usr/sbin/munin20:55:190
1988999248,2cyclictest17050-21kworker/0:021:34:230
1988999245,6cyclictest31830-21kworker/0:300:33:020
1988999241,5cyclictest14579-21munin-run20:10:010
1988999241,5cyclictest14579-21munin-run20:10:010
1988999241,5cyclictest0-21swapper/023:26:580
1988999241,4cyclictest15867-21sadc20:10:210
1988999241,4cyclictest0-21swapper/023:20:460
1988999241,4cyclictest0-21swapper/022:38:130
1988999241,4cyclictest0-21swapper/022:16:480
1988999241,3cyclictest14285-21kworker/u8:322:05:430
1988999241,11cyclictest30802-21/usr/sbin/munin22:00:200
1990699235,3cyclictest0-21swapper/321:26:183
19906992317,3cyclictest16033-21/usr/sbin/munin22:40:133
1990699231,20cyclictest0-21swapper/321:56:023
1990699230,21cyclictest0-21swapper/323:20:463
1990699230,21cyclictest0-21swapper/321:50:183
1990199236,4cyclictest28937-21sshd19:25:272
1990199235,3cyclictest0-21swapper/221:40:082
19901992315,5cyclictest18631-40updatedb00:00:022
1989599237,4cyclictest0-21swapper/122:38:461
1989599233,4cyclictest0-21swapper/123:58:191
1989599232,15cyclictest0-21swapper/123:09:151
1989599232,15cyclictest0-21swapper/122:09:151
1989599231,6cyclictest19815-21kworker/u8:021:47:411
1989599231,6cyclictest0-21swapper/123:31:521
19895992315,3cyclictest0-21swapper/122:18:411
19895992315,2cyclictest0-21swapper/123:00:181
19895992313,3cyclictest0-21swapper/122:30:531
1989599231,11cyclictest0-21swapper/121:39:301
1989599231,10cyclictest0-21swapper/121:14:041
19895992310,3cyclictest25259-40dnf00:15:011
1988999239,3cyclictest26901-21NetworkManager19:21:560
1988999233,2cyclictest23868-21kworker/0:020:38:460
1988999231,4cyclictest0-21swapper/023:03:580
1988999231,11cyclictest5670-21/usr/sbin/munin23:30:210
117562230,3sleep011-21rcuc/020:00:260
19906992220,1cyclictest0-21swapper/323:15:163
19906992216,4cyclictest31049-21grep19:30:263
19906992216,3cyclictest1604-21/usr/sbin/munin19:40:143
1990199229,3cyclictest0-21swapper/220:36:382
1990199228,3cyclictest0-21swapper/223:23:202
1990199226,3cyclictest0-21swapper/223:06:382
1990199224,4cyclictest0-21swapper/221:39:582
1990199222,2cyclictest88950irq/132-enp2s0-20:44:362
1990199221,5cyclictest0-21swapper/221:14:002
1990199221,3cyclictest88950irq/132-enp2s0-00:05:122
1990199221,3cyclictest0-21swapper/223:48:282
1989599229,4cyclictest29517-21memory21:55:181
1989599225,3cyclictest1029-21awk23:20:011
1989599224,3cyclictest0-21swapper/121:52:571
1989599222,15cyclictest0-21swapper/121:16:211
1989599222,15cyclictest0-21swapper/121:16:211
1989599222,12cyclictest17049-21kworker/u8:121:29:491
1989599221,6cyclictest0-21swapper/122:13:591
1989599221,5cyclictest0-21swapper/100:18:441
19895992215,2cyclictest0-21swapper/120:34:111
1989599221,3cyclictest31213-21munin-run23:15:011
1989599221,18cyclictest10396-21taskset20:00:101
1989599221,16cyclictest0-21swapper/119:59:391
1989599221,12cyclictest0-21swapper/100:28:031
1989599221,11cyclictest0-21swapper/123:26:021
19895992210,3cyclictest0-21swapper/100:38:171
1989599220,7cyclictest0-21swapper/122:51:431
1988999228,3cyclictest8203-21/usr/sbin/munin19:55:180
1988999222,9cyclictest18869-21kworker/0:120:21:520
1988999221,6cyclictest0-21swapper/023:35:570
1988999221,6cyclictest0-21swapper/023:35:570
264362211,16sleep226516-21/usr/sbin/munin19:20:232
1990699211,14cyclictest0-21swapper/321:34:313
19906992110,2cyclictest0-21swapper/320:36:373
1990699210,20cyclictest0-21swapper/323:03:573
1990199217,3cyclictest0-21swapper/222:47:282
1990199216,3cyclictest0-21swapper/222:37:232
1990199216,2cyclictest0-21swapper/222:02:132
1990199215,5cyclictest0-21swapper/221:52:372
1990199215,3cyclictest0-21swapper/222:59:002
1990199213,3cyclictest87650irq/129-enp1s0-19:36:452
1990199213,10cyclictest5049-21/usr/sbin/munin22:15:272
1990199211,8cyclictest16586-21perl22:40:132
1990199211,6cyclictest0-21swapper/219:13:512
1990199211,6cyclictest0-21swapper/200:17:162
1990199211,5cyclictest0-21swapper/220:10:582
1990199211,4cyclictest0-21swapper/200:01:202
1989599219,4cyclictest0-21swapper/119:45:121
1989599213,3cyclictest0-21swapper/123:40:501
1989599212,15cyclictest0-21swapper/120:05:481
1989599212,15cyclictest0-21swapper/120:05:481
19895992114,2cyclictest0-21swapper/119:54:321
19895992113,3cyclictest0-21swapper/119:11:511
19895992112,3cyclictest0-21swapper/120:45:061
1989599211,17cyclictest3589-21munin-run19:45:001
1989599211,15cyclictest0-21swapper/121:07:051
1989599211,15cyclictest0-21swapper/119:35:381
1989599211,13cyclictest14285-21kworker/u8:321:41:361
1989599211,11cyclictest22535-21df_inode19:15:031
1989599211,11cyclictest0-21swapper/122:00:141
1989599211,10cyclictest0-21swapper/122:40:021
1989599210,6cyclictest0-21swapper/100:01:171
1989599210,13cyclictest675-21kworker/u8:223:20:501
1988999217,3cyclictest31945-21sshd20:47:270
1988999215,3cyclictest808-21snmpd20:35:020
1988999213,6cyclictest790-21in:imjournal20:20:020
1988999212,9cyclictest28910-21kworker/0:223:15:040
1988999211,6cyclictest0-21swapper/022:40:280
323702201,15sleep1808-21snmpd20:50:041
1990699206,3cyclictest0-21swapper/320:58:033
1990699201,6cyclictest8203-21/usr/sbin/munin19:55:263
19906992014,3cyclictest11060-21gltestperf20:00:173
1990699201,17cyclictest0-21swapper/320:05:173
1990699201,17cyclictest0-21swapper/320:05:163
1990199208,3cyclictest0-21swapper/200:36:132
1990199204,4cyclictest0-21swapper/222:30:492
1990199201,7cyclictest25204-21sshd20:30:272
1990199201,6cyclictest16560-21pmu-power21:25:222
1990199201,5cyclictest0-21swapper/222:25:532
1990199201,5cyclictest0-21swapper/222:25:532
19901992014,3cyclictest573-21tail20:50:162
1989599202,3cyclictest0-21swapper/120:10:501
1989599201,9cyclictest31742-21chrt19:35:011
19895992013,2cyclictest0-21swapper/120:16:521
1989599201,17cyclictest4755-21df21:00:131
1989599201,14cyclictest0-21swapper/120:56:421
1989599201,14cyclictest0-21swapper/120:23:111
1989599201,14cyclictest0-21swapper/119:26:351
1989599201,14cyclictest0-21swapper/119:20:341
1988999204,7cyclictest24537-21kworker/0:119:11:570
1988999202,6cyclictest6290-21kworker/0:121:08:010
1988999201,6cyclictest0-21swapper/020:41:580
1988999200,11cyclictest0-21swapper/020:50:190
45742191,3sleep20-21swapper/222:11:592
1990699191,6cyclictest0-21swapper/320:30:273
1990699191,10cyclictest0-21swapper/320:46:073
1990699191,10cyclictest0-21swapper/319:35:133
1990199196,3cyclictest0-21swapper/223:13:122
1990199191,7cyclictest23283-21/usr/sbin/munin00:10:262
19901991916,2cyclictest0-21swapper/219:17:392
1990199191,5cyclictest0-21swapper/221:57:052
1990199191,5cyclictest0-21swapper/221:49:432
1990199191,5cyclictest0-21swapper/220:00:242
1990199191,4cyclictest0-21swapper/200:21:362
1989599192,16cyclictest0-21swapper/120:40:331
1989599190,11cyclictest0-21swapper/120:25:131
1988999196,3cyclictest87550irq/128-enp1s0-19:30:190
1988999191,9cyclictest31407-21kworker/0:219:42:010
1988999191,6cyclictest31772-21/usr/sbin/munin19:35:240
1988999191,5cyclictest27379-21/usr/sbin/munin19:25:260
1988999191,4cyclictest3807-21/usr/sbin/munin19:45:240
1990699181,8cyclictest0-21swapper/319:49:003
19906991813,3cyclictest15486-21hwlatdetect20:10:153
1990699181,16cyclictest0-21swapper/320:15:133
1990199181,6cyclictest0-21swapper/221:31:242
19901991815,2cyclictest0-21swapper/220:55:122
1990199181,4cyclictest0-21swapper/219:45:122
1990199181,3cyclictest87350irq/126-enp1s0-22:20:182
1990199181,3cyclictest0-21swapper/220:07:582
1990199181,3cyclictest0-21swapper/220:07:582
1990199180,4cyclictest6191-21sshd21:02:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional