You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 08:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Mon Jul 20, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
213462203167,24sleep30-21swapper/319:08:533
213462203167,24sleep30-21swapper/319:08:523
213242202165,25sleep10-21swapper/119:08:351
213242202165,25sleep10-21swapper/119:08:351
213732201164,25sleep00-21swapper/019:09:140
213732201164,25sleep00-21swapper/019:09:140
212672201166,23sleep20-21swapper/219:07:522
212672201166,23sleep20-21swapper/219:07:522
2993721110,3sleep121-21rcuc/119:25:201
2153199103100,2cyclictest0-21swapper/221:00:222
21531998783,3cyclictest0-21swapper/223:06:442
21531997366,3cyclictest0-21swapper/222:52:062
104662720,5sleep230-21rcuc/222:20:272
2153199441,32cyclictest0-21swapper/221:18:102
2151999369,13cyclictest16204-21munin-run20:10:000
2151999369,12cyclictest31601-21awk20:45:010
2153899341,10cyclictest0-21swapper/321:50:533
21525993410,11cyclictest681-21kworker/u8:100:30:381
2153199331,19cyclictest0-21swapper/221:29:422
2153199331,19cyclictest0-21swapper/221:29:422
2153199331,13cyclictest0-21swapper/220:05:272
2153899320,25cyclictest0-21swapper/321:19:213
2153899320,23cyclictest0-21swapper/320:09:053
2153899320,13cyclictest0-21swapper/323:52:573
2153899320,13cyclictest0-21swapper/323:52:573
21531993224,3cyclictest0-21swapper/221:34:462
2153899311,9cyclictest0-21swapper/320:35:053
2153899311,9cyclictest0-21swapper/320:19:253
2153899311,9cyclictest0-21swapper/320:19:253
2153899311,2cyclictest0-21swapper/320:33:093
2153899310,24cyclictest0-21swapper/322:42:573
2153899301,13cyclictest0-21swapper/322:10:523
2153899301,13cyclictest0-21swapper/321:34:213
2153899300,19cyclictest0-21swapper/322:02:123
2153899300,14cyclictest0-21swapper/322:19:243
2153899300,12cyclictest0-21swapper/323:19:053
2153199301,12cyclictest0-21swapper/200:20:552
2153899291,12cyclictest0-21swapper/321:22:573
2153899291,12cyclictest0-21swapper/319:47:243
2153899291,10cyclictest0-21swapper/322:51:093
2153899291,10cyclictest0-21swapper/319:38:013
2153899290,18cyclictest0-21swapper/300:26:443
2153899290,13cyclictest0-21swapper/323:33:533
2153899290,12cyclictest0-21swapper/320:42:173
2153899290,12cyclictest0-21swapper/320:26:293
2153199291,13cyclictest0-21swapper/222:30:352
2153199291,13cyclictest0-21swapper/220:40:132
2153199291,12cyclictest0-21swapper/219:39:552
2153899281,12cyclictest0-21swapper/319:42:163
2153899281,11cyclictest0-21swapper/322:55:293
2153899281,11cyclictest0-21swapper/300:01:133
2153899281,10cyclictest0-21swapper/320:02:053
2153899281,10cyclictest0-21swapper/319:12:293
2153899280,13cyclictest0-21swapper/321:47:133
2153899280,12cyclictest0-21swapper/323:01:173
2153899280,12cyclictest0-21swapper/321:02:013
2153899280,12cyclictest0-21swapper/300:05:293
2153899280,11cyclictest0-21swapper/322:32:293
2153899280,11cyclictest0-21swapper/321:13:293
21531992813,3cyclictest0-21swapper/222:59:112
2153899271,18cyclictest0-21swapper/321:26:013
2153899271,18cyclictest0-21swapper/321:26:013
2153899271,12cyclictest0-21swapper/321:40:493
2153899271,10cyclictest10223-21diskmemload22:09:243
2153899271,10cyclictest0-21swapper/319:34:053
2153899270,12cyclictest0-21swapper/323:46:213
2153899270,11cyclictest0-21swapper/319:53:363
2153899270,11cyclictest0-21swapper/300:36:523
2153899270,11cyclictest0-21swapper/300:20:483
2153899270,11cyclictest0-21swapper/300:14:453
21531992719,3cyclictest0-21swapper/223:24:432
2153199271,19cyclictest0-21swapper/221:37:462
2153899261,7cyclictest0-21swapper/319:57:533
2153899260,9cyclictest0-21swapper/323:35:053
2153899260,17cyclictest0-21swapper/323:26:293
2153899260,12cyclictest0-21swapper/323:40:413
2153899260,12cyclictest0-21swapper/323:07:373
2153899260,11cyclictest0-21swapper/319:26:213
2153899260,11cyclictest0-21swapper/319:21:213
2153899260,10cyclictest0-21swapper/323:57:373
2153899260,10cyclictest0-21swapper/322:48:413
2153899260,10cyclictest0-21swapper/322:21:453
2153899260,10cyclictest0-21swapper/321:38:173
2153199269,4cyclictest31828-21sed23:10:192
21531992616,3cyclictest0-21swapper/200:08:062
21531992611,9cyclictest0-21swapper/223:41:512
2153199261,15cyclictest21545-21ntp_kernel_pll_00:00:222
2153199261,12cyclictest0-21swapper/219:27:222
2153199261,11cyclictest0-21swapper/223:55:422
2153199261,10cyclictest0-21swapper/222:42:512
2151999267,16cyclictest2677-21munin-run23:20:020
2151999261,13cyclictest3639-21awk20:55:020
2151999260,13cyclictest23425-21awk21:40:010
2153899251,10cyclictest0-21swapper/322:27:373
2153899250,9cyclictest0-21swapper/321:56:083
2153899250,8cyclictest0-21swapper/300:32:563
2153899250,11cyclictest0-21swapper/323:14:493
2153899250,10cyclictest0-21swapper/320:51:373
2153899250,10cyclictest0-21swapper/319:15:533
2153199251,7cyclictest0-21swapper/200:17:532
2153199251,6cyclictest0-21swapper/200:30:252
21531992513,3cyclictest0-21swapper/221:22:222
21531992512,7cyclictest0-21swapper/221:52:062
21531992511,4cyclictest0-21swapper/222:10:392
2153199251,13cyclictest6497-21ntp_states19:45:192
2153199251,11cyclictest0-21swapper/222:05:512
2153199250,14cyclictest0-21swapper/222:19:392
2153899241,11cyclictest28898-21taskset00:17:563
2153899240,9cyclictest0-21swapper/322:35:333
2153899240,9cyclictest0-21swapper/322:35:333
2153899240,11cyclictest0-21swapper/320:59:413
2153199249,9cyclictest0-21swapper/223:29:392
2153199247,3cyclictest0-21swapper/220:13:232
21531992411,8cyclictest0-21swapper/222:45:112
2153199241,15cyclictest0-21swapper/220:30:192
2153199241,11cyclictest0-21swapper/223:31:032
2152599241,4cyclictest31595-21perf20:45:011
2152599241,10cyclictest0-21swapper/123:18:061
2151999241,10cyclictest0-21swapper/021:20:250
2153199239,3cyclictest0-21swapper/223:19:032
2153199239,3cyclictest0-21swapper/222:01:472
2153199239,3cyclictest0-21swapper/220:49:272
2153199239,3cyclictest0-21swapper/220:24:112
2153199239,3cyclictest0-21swapper/219:20:582
2153199238,3cyclictest0-21swapper/219:16:462
2153199237,3cyclictest0-21swapper/220:37:072
2153199231,10cyclictest0-21swapper/222:39:172
2153199231,10cyclictest0-21swapper/222:39:172
21531992310,3cyclictest0-21swapper/219:52:032
2152599231,19cyclictest21899-21ntp_states20:20:211
2152599231,19cyclictest10560-21ntp_states23:35:221
2152599231,12cyclictest0-21swapper/122:03:121
2151999238,3cyclictest0-21swapper/023:12:300
2151999236,15cyclictest0-21swapper/019:30:010
2151999231,6cyclictest0-21swapper/022:42:070
2151999231,4cyclictest0-21swapper/021:45:240
2151999231,10cyclictest2649-21sshd19:35:320
2151999230,9cyclictest0-21swapper/019:50:200
2151999230,10cyclictest0-21swapper/023:30:120
305292220,1sleep20-21swapper/221:55:142
2153899225,3cyclictest0-21swapper/320:14:473
2153899221,14cyclictest0-21swapper/320:21:213
2153199228,4cyclictest0-21swapper/222:25:112
2153199228,3cyclictest0-21swapper/200:27:472
2153199227,3cyclictest0-21swapper/223:39:312
2153199226,3cyclictest0-21swapper/221:47:352
21531992210,3cyclictest0-21swapper/220:50:192
2153199220,9cyclictest0-21swapper/219:10:212
2152599229,11cyclictest31639-21kworker/u8:219:10:231
2152599221,3cyclictest7639-21diskstats23:30:131
2152599221,20cyclictest0-21swapper/100:03:101
2152599221,1cyclictest0-21swapper/123:46:201
2152599221,18cyclictest6401-21dnf21:00:161
2152599221,18cyclictest18577-21memory22:40:191
2152599221,18cyclictest0-21swapper/122:45:151
2152599221,10cyclictest0-21swapper/120:15:271
2152599221,10cyclictest0-21swapper/120:15:271
2151999227,5cyclictest26801-21df23:00:150
2151999227,13cyclictest0-21swapper/023:40:010
2151999223,3cyclictest0-21swapper/021:58:400
2151999221,6cyclictest0-21swapper/022:02:160
2151999221,5cyclictest0-21swapper/023:58:410
21519992215,2cyclictest0-21swapper/021:10:190
2151999220,9cyclictest0-21swapper/019:10:320
2153899215,2cyclictest0-21swapper/321:05:253
2153899211,3cyclictest30300-21kworker/u8:023:23:463
2153199218,3cyclictest0-21swapper/220:58:112
2153199216,3cyclictest0-21swapper/221:14:552
2153199216,3cyclictest0-21swapper/220:25:432
2153199216,3cyclictest0-21swapper/200:38:232
2153199211,5cyclictest0-21swapper/223:01:432
2153199211,5cyclictest0-21swapper/200:14:272
2153199211,13cyclictest0-21swapper/220:17:232
2153199211,13cyclictest0-21swapper/220:17:232
2152599218,3cyclictest305302cat21:55:141
2152599216,3cyclictest0-21swapper/121:46:111
2152599212,17cyclictest0-21swapper/122:21:091
2152599211,3cyclictest0-21swapper/121:05:141
2152599211,18cyclictest0-21swapper/122:56:031
2152599211,17cyclictest21798-21entropy21:35:161
2152599211,10cyclictest0-21swapper/122:55:011
2152599210,4cyclictest0-21swapper/100:10:211
2151999214,4cyclictest24485-21/usr/sbin/munin21:40:180
2151999211,9cyclictest0-21swapper/023:05:420
2151999211,3cyclictest0-21swapper/023:25:060
2151999211,11cyclictest4477-21/usr/sbin/munin22:10:180
2151999211,11cyclictest0-21swapper/021:17:390
2151999211,10cyclictest0-21swapper/021:30:500
2151999210,6cyclictest0-21swapper/023:24:520
2151999210,6cyclictest0-21swapper/022:47:400
2151999210,6cyclictest0-21swapper/022:06:290
2151999210,5cyclictest0-21swapper/022:33:280
323852200,3sleep00-21swapper/000:25:190
2153899201,8cyclictest0-21swapper/320:46:593
2153199207,8cyclictest0-21swapper/223:46:382
2153199201,12cyclictest0-21swapper/219:59:552
2153199200,13cyclictest0-21swapper/221:06:072
2152599203,8cyclictest18245-1kworker/u9:200:36:101
2152599202,3cyclictest0-21swapper/122:15:131
2152599202,17cyclictest0-21swapper/123:42:481
2152599202,17cyclictest0-21swapper/123:08:121
2152599202,17cyclictest0-21swapper/119:43:091
2152599202,10cyclictest0-21swapper/120:46:301
2152599201,9cyclictest0-21swapper/123:11:421
2152599201,7cyclictest0-21swapper/121:25:561
2152599201,7cyclictest0-21swapper/121:25:561
21525992017,2cyclictest0-21swapper/122:35:191
21525992017,2cyclictest0-21swapper/122:35:191
2152599201,6cyclictest1680-21expr19:35:181
21525992016,3cyclictest0-21swapper/123:21:381
2152599201,2cyclictest24893-21mailstats21:40:231
2152599201,18cyclictest0-21swapper/122:25:151
2152599201,18cyclictest0-21swapper/119:21:301
2152599201,17cyclictest16210-21perf20:10:011
2152599201,16cyclictest31786-21perl19:30:171
2152599201,16cyclictest26463-21/usr/sbin/munin23:00:171
2152599201,10cyclictest24960-21diskstats19:15:151
2152599201,10cyclictest0-21swapper/121:32:551
2152599201,10cyclictest0-21swapper/100:20:261
2152599200,7cyclictest0-21swapper/121:14:261
2152599200,4cyclictest0-21swapper/123:29:441
2152599200,3cyclictest0-21swapper/123:56:121
2152599200,15cyclictest0-21swapper/100:29:191
2151999207,4cyclictest18588-21unixbench_singl21:25:250
2151999207,4cyclictest18588-21unixbench_singl21:25:250
2151999206,3cyclictest9832-21/usr/sbin/munin19:55:160
2151999206,3cyclictest10851-21munin-run22:25:010
2151999202,10cyclictest5429-21/usr/sbin/munin19:45:110
2151999201,9cyclictest0-21swapper/000:21:510
2151999201,9cyclictest0-21swapper/000:16:040
2151999201,9cyclictest0-21swapper/000:09:340
2151999201,7cyclictest15466-21/usr/sbin/munin22:35:230
2151999201,7cyclictest15466-21/usr/sbin/munin22:35:220
2151999201,3cyclictest0-21swapper/022:59:250
2151999201,2cyclictest0-21swapper/000:37:480
2151999201,17cyclictest0-21swapper/021:00:350
2151999201,11cyclictest6677-21/usr/sbin/munin22:15:220
2151999201,10cyclictest0-21swapper/000:30:440
2153199197,3cyclictest0-21swapper/220:01:072
2153199195,3cyclictest0-21swapper/221:40:102
2152599197,9cyclictest28010-21load00:15:191
2152599193,2cyclictest87550irq/128-enp1s0-20:25:281
2152599192,3cyclictest0-21swapper/120:11:481
2152599192,16cyclictest0-21swapper/121:17:331
2152599191,9cyclictest0-21swapper/122:10:541
2152599191,7cyclictest22706-21/usr/sbin/munin00:05:241
2152599191,7cyclictest0-21swapper/121:50:371
2152599191,6cyclictest0-21swapper/123:54:041
2152599191,6cyclictest0-21swapper/123:54:041
21525991915,3cyclictest0-21swapper/120:36:321
21525991915,3cyclictest0-21swapper/119:58:061
2152599191,3cyclictest0-21swapper/120:54:451
2152599190,18cyclictest0-21swapper/122:05:021
2152599190,17cyclictest0-21swapper/120:00:121
2151999199,7cyclictest15610-21systemd-journal23:47:040
2151999193,2cyclictest0-21swapper/020:18:300
2151999193,2cyclictest0-21swapper/020:18:300
2151999191,9cyclictest0-21swapper/000:12:450
2151999191,6cyclictest0-21swapper/022:26:230
2151999191,11cyclictest25225-21/usr/sbin/munin20:30:250
2151999190,13cyclictest0-21swapper/019:21:310
162642190,4sleep12152599cyclictest21:20:341
2153199184,3cyclictest0-21swapper/219:40:112
2153199180,9cyclictest0-21swapper/219:32:212
2153199180,11cyclictest0-21swapper/223:50:142
2153199180,11cyclictest0-21swapper/223:50:142
2152599181,16cyclictest0-21swapper/122:32:591
2152599181,16cyclictest0-21swapper/120:57:411
2152599181,16cyclictest0-21swapper/120:31:441
2152599181,16cyclictest0-21swapper/119:47:411
2151999182,6cyclictest24790-21sshd20:27:300
2151999182,3cyclictest20562-21cpuspeed_turbos00:00:140
2151999181,8cyclictest0-21swapper/023:54:220
2151999181,8cyclictest0-21swapper/023:54:220
2151999181,6cyclictest3853-21/usr/sbin/munin20:55:180
2151999181,6cyclictest0-21swapper/020:46:020
2152599170,16cyclictest0-21swapper/119:51:061
2151999177,7cyclictest32196-21needreboot19:30:210
2151999171,9cyclictest87350irq/126-enp1s0-21:52:340
2151999171,6cyclictest18245-1kworker/u9:220:12:300
2151999171,6cyclictest0-21swapper/023:40:340
2151999171,5cyclictest12026-21/usr/sbin/munin20:00:120
2151999171,4cyclictest0-21swapper/020:35:260
2151999171,3cyclictest24855-21cstates19:15:140
2151999170,6cyclictest0-21swapper/022:50:260
2151999161,6cyclictest0-21swapper/021:05:160
2151999161,5cyclictest0-21swapper/019:43:560
2151999161,3cyclictest20820-21/usr/sbin/munin20:20:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional