You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 03:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Fri Jun 05, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
92942244161,70sleep00-21swapper/019:07:000
93952232164,55sleep30-21swapper/319:08:193
93492203165,25sleep20-21swapper/219:07:412
95022202166,24sleep10-21swapper/119:09:411
876121500,1sleep00-21swapper/021:30:250
9624998177,2cyclictest8443-21munin-run00:00:022
274772560,1sleep00-21swapper/022:15:130
9624995551,3cyclictest0-21swapper/221:42:012
139542540,5sleep1961799cyclictest19:15:231
222222520,5sleep322223-21grep20:50:113
315372500,4sleep011-21rcuc/019:55:230
170752490,1sleep10-21swapper/119:25:011
156902440,2sleep20-21swapper/219:20:182
961799351,8cyclictest0-21swapper/100:28:331
961799341,16cyclictest0-21swapper/121:58:211
961799341,14cyclictest0-21swapper/122:37:461
962899331,25cyclictest0-21swapper/322:50:033
962899321,9cyclictest0-21swapper/319:25:163
962899320,12cyclictest0-21swapper/300:30:193
961799321,8cyclictest0-21swapper/122:23:211
961799321,8cyclictest0-21swapper/100:17:051
961799321,13cyclictest6828-21ntp_states22:40:221
961799321,13cyclictest0-21swapper/121:21:331
961199328,12cyclictest30692-21munin-run21:10:000
962899310,9cyclictest0-21swapper/322:25:243
962899310,9cyclictest0-21swapper/322:25:233
961799311,8cyclictest0-21swapper/123:46:491
961799311,8cyclictest0-21swapper/123:37:371
961799311,8cyclictest0-21swapper/122:51:581
961799311,5cyclictest0-21swapper/120:57:021
961799311,16cyclictest0-21swapper/123:26:371
961799311,13cyclictest0-21swapper/121:39:131
961199318,11cyclictest31954-21awk23:40:000
962899301,19cyclictest0-21swapper/320:24:273
961799301,6cyclictest0-21swapper/100:30:171
961799301,12cyclictest0-21swapper/123:02:461
961799301,12cyclictest0-21swapper/120:07:171
961799301,12cyclictest0-21swapper/120:07:171
961799301,11cyclictest0-21swapper/120:35:181
961799301,11cyclictest0-21swapper/119:35:011
961199304,12cyclictest15078-21awk00:15:020
961199301,14cyclictest7109-21sadc21:30:010
962899292,10cyclictest20871-21df_inode22:00:163
962899291,12cyclictest0-21swapper/321:02:313
962899291,11cyclictest0-21swapper/321:37:003
962899291,11cyclictest0-21swapper/320:57:033
962899291,11cyclictest0-21swapper/300:29:433
962899290,11cyclictest0-21swapper/320:19:593
961799291,9cyclictest0-21swapper/123:16:581
961799291,11cyclictest25998-21ntp_states22:10:211
962899281,9cyclictest0-21swapper/319:24:083
962899281,11cyclictest0-21swapper/320:26:113
962899281,10cyclictest30681-21diskmemload00:01:523
962899280,22cyclictest0-21swapper/320:47:153
962899280,11cyclictest0-21swapper/322:21:043
961799285,12cyclictest0-21swapper/122:55:101
961799281,9cyclictest0-21swapper/122:00:491
961799281,13cyclictest0-21swapper/122:25:171
961799281,13cyclictest0-21swapper/122:25:171
961799281,13cyclictest0-21swapper/121:05:301
961799281,13cyclictest0-21swapper/120:30:181
961799281,13cyclictest0-21swapper/100:03:091
961799281,12cyclictest0-21swapper/119:48:171
961799281,11cyclictest0-21swapper/123:59:451
961799281,10cyclictest28428-21diskstats19:50:131
961799281,10cyclictest0-21swapper/121:01:381
961799280,5cyclictest0-21swapper/122:08:171
961199289,16cyclictest5550-21idleruntime-cro22:40:020
962899271,12cyclictest0-21swapper/321:57:203
962899271,12cyclictest0-21swapper/319:58:483
962899271,11cyclictest0-21swapper/323:56:403
962899271,10cyclictest15474-21memory23:00:193
962899271,10cyclictest0-21swapper/322:57:233
962899270,2cyclictest0-21swapper/319:39:443
962899270,17cyclictest0-21swapper/320:32:473
962899270,10cyclictest0-21swapper/319:15:393
961799271,9cyclictest0-21swapper/121:33:451
961799271,6cyclictest0-21swapper/121:45:011
961799271,17cyclictest0-21swapper/123:13:421
961799271,11cyclictest0-21swapper/121:46:091
961799271,11cyclictest0-21swapper/121:28:571
961799271,10cyclictest0-21swapper/123:30:211
961799271,10cyclictest0-21swapper/123:07:501
961799271,10cyclictest0-21swapper/120:22:541
961799271,10cyclictest0-21swapper/120:04:591
961799271,10cyclictest0-21swapper/100:38:381
961799271,10cyclictest0-21swapper/100:22:531
962899261,8cyclictest0-21swapper/322:35:153
962899261,2cyclictest0-21swapper/320:03:123
962899261,11cyclictest0-21swapper/323:22:513
962899261,10cyclictest0-21swapper/320:41:153
962899260,9cyclictest0-21swapper/323:25:233
962899260,9cyclictest0-21swapper/321:12:483
962899260,10cyclictest0-21swapper/322:48:273
962899260,10cyclictest0-21swapper/320:06:323
962899260,10cyclictest0-21swapper/320:06:323
961799261,9cyclictest0-21swapper/119:55:451
961799261,9cyclictest0-21swapper/100:09:371
961799261,11cyclictest0-21swapper/123:53:251
961799261,11cyclictest0-21swapper/123:40:371
961799261,11cyclictest0-21swapper/120:52:061
961799261,10cyclictest0-21swapper/123:22:021
961799261,10cyclictest0-21swapper/122:33:021
961799261,10cyclictest0-21swapper/120:46:101
962899255,2cyclictest0-21swapper/300:20:283
962899251,15cyclictest0-21swapper/321:05:513
962899251,12cyclictest0-21swapper/322:16:273
962899251,11cyclictest0-21swapper/319:41:003
962899251,11cyclictest0-21swapper/300:14:323
962899251,10cyclictest0-21swapper/319:34:483
962899251,10cyclictest0-21swapper/300:38:433
962899250,9cyclictest0-21swapper/323:40:243
962899250,9cyclictest0-21swapper/319:46:003
961799251,12cyclictest0-21swapper/122:49:541
961799251,11cyclictest0-21swapper/120:18:021
961799251,10cyclictest0-21swapper/121:13:051
961199252,11cyclictest20318-21perf22:00:010
961199252,11cyclictest19514-21basename00:25:010
9611992512,3cyclictest28860-21pmu-power23:30:220
962899248,3cyclictest0-21swapper/321:29:083
962899248,3cyclictest0-21swapper/321:24:003
962899247,9cyclictest0-21swapper/320:37:413
962899241,9cyclictest1706-21df22:30:133
962899241,11cyclictest0-21swapper/323:50:083
962899241,11cyclictest0-21swapper/323:05:233
962899240,9cyclictest0-21swapper/319:10:513
962899240,22cyclictest0-21swapper/321:34:363
962899240,11cyclictest0-21swapper/319:51:323
962899240,10cyclictest0-21swapper/323:36:203
962899240,10cyclictest0-21swapper/321:18:403
962899240,10cyclictest0-21swapper/300:16:123
962499242,11cyclictest6132-21strings21:25:212
962499241,5cyclictest0-21swapper/223:26:192
961799241,15cyclictest0-21swapper/119:44:291
961799241,13cyclictest0-21swapper/119:13:101
961799241,11cyclictest0-21swapper/121:18:131
961799241,10cyclictest0-21swapper/121:54:451
961799241,10cyclictest0-21swapper/100:14:011
961199243,6cyclictest14106-40dnf22:57:250
962899238,9cyclictest0-21swapper/323:31:563
962899234,9cyclictest0-21swapper/321:41:023
962899231,9cyclictest30681-21diskmemload21:46:263
962899231,14cyclictest0-21swapper/321:51:403
962899231,10cyclictest0-21swapper/323:16:353
962899230,9cyclictest0-21swapper/323:47:403
961799238,3cyclictest4856-21df20:10:111
961799231,9cyclictest0-21swapper/120:28:301
961799231,10cyclictest19939-21kworker/u8:122:15:051
961199237,6cyclictest0-21swapper/022:45:110
961199237,6cyclictest0-21swapper/000:03:240
962899225,3cyclictest0-21swapper/300:08:363
962899221,9cyclictest0-21swapper/322:13:463
962899221,9cyclictest0-21swapper/322:05:143
961799228,1cyclictest0-21swapper/119:29:071
961199227,6cyclictest0-21swapper/021:15:450
961199225,15cyclictest11797-21awk19:15:020
962899217,9cyclictest0-21swapper/323:14:413
962899215,2cyclictest4523-21/usr/sbin/munin20:10:153
962499215,4cyclictest23364-21kworker/u8:221:01:042
962499211,6cyclictest0-21swapper/223:45:072
962499211,6cyclictest0-21swapper/222:59:292
962499207,4cyclictest24655-1kworker/u9:023:02:112
962499204,5cyclictest8810-21tr00:00:112
962499202,7cyclictest18679-21diskstats21:55:152
962499202,10cyclictest763-21dbus-daemon19:57:282
962499201,7cyclictest0-21swapper/223:40:232
962499201,7cyclictest0-21swapper/221:10:052
9624992014,3cyclictest8615-21sed22:45:162
962499201,13cyclictest0-21swapper/200:31:422
961799201,14cyclictest0-21swapper/120:44:541
961799201,11cyclictest21838-21diskstats19:35:131
961199205,6cyclictest0-21swapper/023:50:400
961199205,6cyclictest0-21swapper/023:06:070
961199202,10cyclictest14561-21/usr/sbin/munin23:00:220
961199201,5cyclictest24655-1kworker/u9:022:12:100
961199201,4cyclictest6503-21perf20:15:020
961199201,16cyclictest10676-21fschecks_time22:50:150
961199201,10cyclictest8459-40updatedb00:00:010
961199201,10cyclictest11492-21tail00:05:180
9611992010,3cyclictest0-21swapper/023:26:180
961199200,9cyclictest0-21swapper/022:21:420
313992200,2sleep230-21rcuc/223:35:262
962899191,6cyclictest0-21swapper/322:41:553
962499194,6cyclictest32-21ksoftirqd/223:50:262
962499191,8cyclictest4221-21gltestperf22:35:152
962499191,7cyclictest16584-21switchtime00:15:252
962499191,6cyclictest0-21swapper/223:12:362
962499191,5cyclictest24655-1kworker/u9:022:13:292
9624991912,4cyclictest3161-21tr20:05:182
9624991912,4cyclictest3161-21tr20:05:182
961199198,3cyclictest23477-21awk19:40:010
961199197,3cyclictest0-21swapper/023:49:230
961199196,5cyclictest0-21swapper/023:10:310
961199195,4cyclictest0-21swapper/022:03:250
961199194,4cyclictest19415-21kworker/u8:200:30:130
961199192,15cyclictest24718-21perf22:10:010
961199191,4cyclictest10791-21kworker/u8:221:43:280
961199191,2cyclictest87350irq/126-enp1s0-21:53:130
9611991912,4cyclictest28501-21grep19:50:150
961199191,10cyclictest30912-21/usr/sbin/munin21:10:170
961199191,10cyclictest0-21swapper/022:40:210
961199190,9cyclictest0-21swapper/000:16:140
962499185,6cyclictest0-21swapper/200:38:232
962499185,4cyclictest15026-21runrttasks00:14:392
962499184,3cyclictest15486-21head21:45:262
962499182,7cyclictest10889-21/usr/sbin/munin00:05:272
962499182,3cyclictest0-21swapper/222:43:182
962499182,3cyclictest0-21swapper/220:57:352
962499182,15cyclictest0-21swapper/221:20:572
962499182,14cyclictest0-21swapper/219:43:192
962499181,7cyclictest7790-21ntp_states20:15:192
962499181,7cyclictest0-21swapper/223:08:052
962499181,7cyclictest0-21swapper/220:10:112
962499181,7cyclictest0-21swapper/219:45:242
962499181,6cyclictest0-21swapper/200:25:332
962499181,5cyclictest24655-1kworker/u9:020:43:042
962499181,4cyclictest28715-21missed_timers23:30:202
9624991812,3cyclictest9700-21if_err_enp1s020:20:162
962499181,14cyclictest1094-21strings20:00:202
9624991811,4cyclictest0-21swapper/222:50:252
962499181,13cyclictest13966-21gltestperf20:30:152
961199187,3cyclictest13937-21/usr/sbin/munin21:45:260
961199186,3cyclictest87350irq/126-enp1s0-19:45:250
961199184,5cyclictest0-21swapper/021:38:200
961199184,2cyclictest87350irq/126-enp1s0-00:25:220
961199182,9cyclictest25238-21awk00:35:240
961199182,3cyclictest0-21swapper/023:19:370
961199181,5cyclictest0-21swapper/022:32:190
70862180,3sleep011-21rcuc/020:15:140
45872181,2sleep00-21swapper/021:20:550
178122181,3sleep20-21swapper/221:51:272
168452181,2sleep00-21swapper/019:24:490
962499176,3cyclictest87650irq/129-enp1s0-00:20:192
962499171,7cyclictest29988-21switchtime21:05:252
962499171,7cyclictest0-21swapper/222:04:162
962499171,3cyclictest1734-21sed21:15:212
9624991713,3cyclictest0-21swapper/221:38:322
9624991712,4cyclictest0-21swapper/219:35:212
9624991712,3cyclictest7445-21tr21:30:132
962499171,15cyclictest0-21swapper/219:31:202
962499171,14cyclictest0-21swapper/220:45:452
9624991711,3cyclictest827-21runrttasks22:09:342
962499171,13cyclictest30861-21grep22:20:242
962499171,13cyclictest18666-21grep19:25:252
9624991711,3cyclictest11929-21munin-node19:15:052
9624991711,3cyclictest10757-21missed_timers19:10:182
962499170,6cyclictest0-21swapper/220:28:592
961199176,3cyclictest23537-21cstates23:20:120
9611991711,4cyclictest32373-21/usr/sbin/munin22:25:190
9611991711,4cyclictest32373-21/usr/sbin/munin22:25:190
961199171,14cyclictest13693-21strings19:15:180
961199171,14cyclictest0-21swapper/020:50:590
9611991711,3cyclictest2910-21perl20:05:160
9611991711,3cyclictest2910-21perl20:05:160
961199171,13cyclictest19297-21/usr/sbin/munin19:30:130
961199171,10cyclictest0-21swapper/019:44:300
323352171,2sleep20-21swapper/222:25:192
323352171,2sleep20-21swapper/222:25:192
196512171,3sleep00-21swapper/020:44:480
962499161,9cyclictest0-21swapper/223:23:312
962499161,7cyclictest0-21swapper/223:15:332
962499161,4cyclictest0-21swapper/220:35:182
9624991612,3cyclictest0-21swapper/222:15:482
9624991612,3cyclictest0-21swapper/220:50:292
9624991611,3cyclictest1960-21tail22:30:152
961199162,6cyclictest839-21lldpd21:00:520
961199161,8cyclictest1888-21NetworkManager20:02:230
961199161,6cyclictest9260-21df_inode20:20:140
961199161,6cyclictest32166-21/usr/sbin/munin23:40:200
961199161,4cyclictest17098-21/usr/sbin/munin19:25:140
961199161,14cyclictest0-21swapper/020:25:430
9611991611,3cyclictest15852-21sh20:35:140
297942161,2sleep20-21swapper/219:51:482
205832161,2sleep00-21swapper/020:45:170
150612161,2sleep00-21swapper/020:32:160
9611991510,3cyclictest24300-21/usr/sbin/munin20:55:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional