You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 04:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Sep 18, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25869992726,1cyclictest62550irq/127-eno123:41:400
25877992621,4cyclictest19510-21diskmemload22:22:452
2588299250,24cyclictest4812-21cron00:20:013
2588299250,24cyclictest0-21swapper/323:32:133
25877992520,4cyclictest14239-21sh21:33:462
2587799250,24cyclictest30128-21sh21:47:442
2587799250,24cyclictest14857-21awk00:00:002
2587799250,24cyclictest0-21swapper/223:23:072
2587799250,21cyclictest8949-21ssh21:28:592
2587799250,1cyclictest19510-21diskmemload21:23:002
2587799250,1cyclictest17205-21ssh22:06:182
2587299250,24cyclictest27580-21ssh21:46:371
2587299250,24cyclictest0-21swapper/123:28:301
2587299250,1cyclictest31748-21cron00:15:011
25869992522,2cyclictest22530-21/usr/sbin/munin23:37:400
2586999250,24cyclictest24347-21sh23:10:270
2586999250,24cyclictest19510-21diskmemload22:08:050
2586999250,24cyclictest16968-21apt-get00:02:350
2586999250,24cyclictest0-21swapper/000:26:580
25882992422,1cyclictest0-21swapper/322:34:243
2588299240,23cyclictest0-21swapper/323:55:593
2588299240,23cyclictest0-21swapper/323:23:123
2588299240,23cyclictest0-21swapper/323:13:033
2588299240,0cyclictest19510-21diskmemload00:32:403
25877992422,2cyclictest0-21swapper/223:12:122
25877992422,1cyclictest19510-21diskmemload22:27:402
25877992420,3cyclictest19510-21diskmemload23:07:282
25877992420,3cyclictest19510-21diskmemload00:05:502
2587799240,23cyclictest0-21swapper/221:14:072
2587799240,20cyclictest19510-21diskmemload21:35:142
2587799240,20cyclictest0-21swapper/223:40:062
2587799240,1cyclictest0-21swapper/222:02:522
2587799240,0cyclictest0-21swapper/223:35:302
2587799240,0cyclictest0-21swapper/222:35:052
2587799240,0cyclictest0-21swapper/221:18:002
25872992424,0cyclictest0-21swapper/122:55:101
25872992423,1cyclictest0-21swapper/123:54:021
2587299240,24cyclictest0-21swapper/121:14:311
2587299240,23cyclictest0-21swapper/123:48:551
2587299240,0cyclictest0-21swapper/121:15:051
25869992423,1cyclictest62550irq/127-eno100:37:450
25869992420,3cyclictest28228-21rm22:15:360
2586999241,2cyclictest26224-21cron21:45:010
2586999240,3cyclictest2615-21dump-pmu-power23:20:010
2586999240,23cyclictest31823-21ssh23:46:200
2586999240,23cyclictest19510-21diskmemload22:30:390
2586999240,1cyclictest6283-21ssh21:27:340
2586999240,1cyclictest4395-21ssh00:19:290
25882992323,0cyclictest0-21swapper/323:06:593
25882992323,0cyclictest0-21swapper/322:45:283
25882992323,0cyclictest0-21swapper/322:02:563
25882992322,1cyclictest0-21swapper/300:13:063
25882992321,1cyclictest1115-21kernelversion23:47:383
25882992321,1cyclictest0-21swapper/323:20:013
25882992321,1cyclictest0-21swapper/322:55:203
25882992321,1cyclictest0-21swapper/322:52:173
25882992321,1cyclictest0-21swapper/322:18:073
25882992321,1cyclictest0-21swapper/321:23:053
25882992320,2cyclictest9088-21hddtemp_smartct22:27:373
25882992320,2cyclictest10689-21latency_hist19:45:003
2588299230,23cyclictest0-21swapper/323:43:193
2588299230,23cyclictest0-21swapper/322:20:123
2588299230,23cyclictest0-21swapper/321:53:073
2588299230,23cyclictest0-21swapper/321:12:203
2588299230,22cyclictest32731-21turbostat.cron20:30:003
2588299230,22cyclictest27792-21/usr/sbin/munin21:17:353
2588299230,22cyclictest17530-21grep00:02:373
2588299230,22cyclictest0-21swapper/321:50:013
2588299230,22cyclictest0-21swapper/320:02:333
2588299230,1cyclictest19510-21diskmemload22:09:373
2588299230,0cyclictest19510-21diskmemload23:50:033
2588299230,0cyclictest19510-21diskmemload22:43:523
2588299230,0cyclictest19510-21diskmemload21:55:333
25877992323,0cyclictest19510-21diskmemload23:50:162
25877992323,0cyclictest19510-21diskmemload00:01:562
25877992323,0cyclictest0-21swapper/222:42:132
25877992321,2cyclictest0-21swapper/222:50:542
25877992320,2cyclictest0-21swapper/222:10:442
2587799230,23cyclictest0-21swapper/222:16:562
2587799230,23cyclictest0-21swapper/200:31:052
2587799230,22cyclictest22316-21ssh21:41:562
2587799230,22cyclictest1109-21sh00:17:072
2587799230,22cyclictest0-21swapper/223:01:522
2587799230,1cyclictest0-21swapper/222:45:562
2587799230,0cyclictest0-21swapper/223:27:552
2587799230,0cyclictest0-21swapper/221:53:512
2587799230,0cyclictest0-21swapper/200:24:272
25872992323,0cyclictest19510-21diskmemload23:41:021
25872992323,0cyclictest0-21swapper/123:04:461
25872992323,0cyclictest0-21swapper/121:56:551
25872992323,0cyclictest0-21swapper/100:00:471
25872992321,1cyclictest0-21swapper/122:26:531
2587299230,23cyclictest0-21swapper/123:35:151
2587299230,23cyclictest0-21swapper/122:40:221
2587299230,22cyclictest6315-21munin-run22:25:011
2587299230,22cyclictest21908-21ssh00:35:371
2587299230,22cyclictest21177-21ssh00:05:171
2587299230,22cyclictest20226-21ssh00:33:331
2587299230,22cyclictest16298-21ssh22:05:041
2587299230,1cyclictest30703-21munin-run19:20:011
2587299230,1cyclictest28922-21ssh22:45:071
2587299230,0cyclictest0-21swapper/123:15:141
2587299230,0cyclictest0-21swapper/122:54:491
2587299230,0cyclictest0-21swapper/122:17:131
2587299230,0cyclictest0-21swapper/121:32:191
25869992323,0cyclictest28343-21apt-get20:22:340
25869992323,0cyclictest19510-21diskmemload23:01:560
25869992323,0cyclictest19510-21diskmemload22:54:060
25869992323,0cyclictest0-21swapper/023:50:180
25869992323,0cyclictest0-21swapper/022:57:450
25869992323,0cyclictest0-21swapper/000:22:070
25869992321,2cyclictest867-21sh21:51:500
25869992321,1cyclictest9842-21tr22:27:420
25869992321,1cyclictest0-21swapper/022:14:290
25869992321,1cyclictest0-21swapper/021:58:340
25869992320,2cyclictest21687-21ssh00:05:580
2586999230,23cyclictest0-21swapper/023:57:070
2586999230,23cyclictest0-21swapper/023:05:450
2586999230,23cyclictest0-21swapper/021:10:280
2586999230,22cyclictest9989-21sh23:27:200
2586999230,22cyclictest0-21swapper/021:22:360
2586999230,1cyclictest15813-21ssh21:35:350
2586999230,0cyclictest19510-21diskmemload21:47:520
2586999230,0cyclictest0-21swapper/023:21:440
25882992222,0cyclictest0-21swapper/321:31:443
25882992222,0cyclictest0-21swapper/300:05:453
25882992221,1cyclictest23523-21apt-get20:12:343
25882992221,0cyclictest19028-21apt-get23:35:013
25882992220,1cyclictest2733-21munin-run20:35:003
25882992220,1cyclictest0-21swapper/323:26:353
25882992220,1cyclictest0-21swapper/323:00:173
2588299220,21cyclictest3926-21apt-get19:32:343
2588299220,21cyclictest19510-21diskmemload00:20:553
2588299220,21cyclictest0-21swapper/321:25:223
2588299220,21cyclictest0-21swapper/319:37:333
2588299220,21cyclictest0-21swapper/319:22:333
2588299220,21cyclictest0-21swapper/300:35:313
2588299220,1cyclictest28728-21apt-get19:17:323
2588299220,1cyclictest21128-21sh21:40:173
2588299220,1cyclictest19510-21diskmemload22:12:043
2588299220,1cyclictest17010-21ssh21:37:173
2588299220,1cyclictest12775-21apt-get20:57:343
2588299220,1cyclictest0-21swapper/300:25:113
25877992222,0cyclictest0-21swapper/222:31:262
25877992222,0cyclictest0-21swapper/220:17:332
25877992222,0cyclictest0-21swapper/200:27:072
25877992220,1cyclictest19510-21diskmemload23:30:402
2587799220,22cyclictest19510-21diskmemload00:35:322
2587799220,22cyclictest0-21swapper/222:55:082
2587799220,21cyclictest8290-21latency_hist19:40:002
2587799220,1cyclictest20280-21apt-get20:05:012
2587799220,1cyclictest14752-21sendmail-msp21:00:012
2587799220,1cyclictest0-21swapper/220:05:022
2587799220,0cyclictest0-21swapper/223:46:112
25872992222,0cyclictest19510-21diskmemload22:36:591
25872992222,0cyclictest0-21swapper/123:11:341
25872992222,0cyclictest0-21swapper/122:10:451
25872992222,0cyclictest0-21swapper/100:19:011
25872992220,2cyclictest12539-21ssh22:30:401
25872992220,1cyclictest2205-21ptp4l-jitter21:22:411
25872992220,1cyclictest13202-21sed00:27:401
2587299220,22cyclictest19510-21diskmemload22:04:381
2587299220,21cyclictest5365-21ssh21:26:141
2587299220,21cyclictest14229-21ssh23:30:361
2587299220,21cyclictest0-21swapper/121:36:421
2587299220,1cyclictest9939-21perf20:50:011
2587299220,1cyclictest6637-21ssh00:22:141
2587299220,1cyclictest18648-21ssh23:05:151
2587299220,1cyclictest0-21swapper/123:22:151
2587299220,1cyclictest0-21swapper/121:50:211
2587299220,0cyclictest0-21swapper/123:58:001
2587299220,0cyclictest0-21swapper/119:37:331
25869992222,0cyclictest19510-21diskmemload22:46:560
25869992220,1cyclictest3171-21apt-get20:37:330
25869992220,1cyclictest23518-21ssh22:40:220
25869992220,1cyclictest16179-21ssh00:30:180
2586999220,21cyclictest0-21swapper/023:31:240
2586999220,21cyclictest0-21swapper/022:35:360
2586999220,21cyclictest0-21swapper/022:20:570
2586999220,1cyclictest3482-21munin-run19:30:010
2586999220,1cyclictest27223-21ssh21:17:100
2586999220,1cyclictest11943-21ssh22:01:490
2586999220,0cyclictest0-21swapper/021:30:210
25882992119,1cyclictest646-21apt-get19:25:023
25882992118,2cyclictest313-21systemd-journal20:19:563
25877992121,0cyclictest0-21swapper/221:55:282
25877992120,1cyclictest7985-21apt-get20:47:342
25877992119,1cyclictest8738-21apt-get19:42:332
2587799210,20cyclictest31146-21apt-get19:22:332
2587799210,20cyclictest0-21swapper/219:32:342
2587799210,0cyclictest0-21swapper/219:17:322
25872992121,0cyclictest0-21swapper/121:42:101
25872992121,0cyclictest0-21swapper/119:32:341
25872992119,1cyclictest5146-21munin-run20:40:001
2587299210,20cyclictest1512-21apt-get19:27:331
2587299210,20cyclictest0-21swapper/120:57:341
2586999210,20cyclictest0-21swapper/020:32:330
25882992020,0cyclictest0-21swapper/320:05:023
2588299200,19cyclictest0-21swapper/321:10:003
25877992020,0cyclictest26277-21apt-get19:12:322
25877992020,0cyclictest0-21swapper/220:22:342
2587799200,19cyclictest0-21swapper/220:27:322
2587799200,0cyclictest0-21swapper/219:57:332
2587799200,0cyclictest0-21swapper/219:27:332
25872992020,0cyclictest0-21swapper/120:22:341
25872992020,0cyclictest0-21swapper/120:17:331
25872992020,0cyclictest0-21swapper/119:52:331
25872992019,0cyclictest755-21apt-get20:32:331
25872992018,1cyclictest32750-21idleruntime-cro20:30:001
25869992020,0cyclictest11134-21apt-get19:47:330
25869992020,0cyclictest0-21swapper/020:52:340
25869992020,0cyclictest0-21swapper/020:17:330
25869992020,0cyclictest0-21swapper/019:12:320
2586999200,0cyclictest0-21swapper/021:02:330
2586999200,0cyclictest0-21swapper/020:27:320
2586999200,0cyclictest0-21swapper/020:12:340
2586999200,0cyclictest0-21swapper/020:02:330
2586999200,0cyclictest0-21swapper/019:52:330
2586999200,0cyclictest0-21swapper/019:37:330
25882991919,0cyclictest0-21swapper/320:37:333
25877991919,0cyclictest0-21swapper/221:02:332
25877991913,5cyclictest25451-21cron20:15:012
2587799190,0cyclictest0-21swapper/220:32:332
25872991919,0cyclictest0-21swapper/121:07:341
25869991919,0cyclictest0-21swapper/020:57:340
25869991918,1cyclictest0-21swapper/021:07:340
25869991917,1cyclictest646-21apt-get19:25:010
25877991818,0cyclictest5593-21apt-get20:42:342
25877991818,0cyclictest0-21swapper/221:07:342
25877991818,0cyclictest0-21swapper/219:47:332
25869991814,3cyclictest0-21swapper/019:34:550
25882991717,0cyclictest0-21swapper/320:47:343
25872991716,1cyclictest0-21swapper/120:05:021
25872991715,1cyclictest13073-21turbostat.cron19:50:001
25872991714,2cyclictest17136-21latency_hist21:05:011
25869991717,0cyclictest0-21swapper/020:47:350
25869991717,0cyclictest0-21swapper/019:17:320
25869991715,1cyclictest20280-21apt-get20:05:020
25882991616,0cyclictest0-21swapper/320:22:343
25882991615,1cyclictest17868-21turbostat.cron20:00:003
2588299160,15cyclictest13055-21awk19:50:003
2587799160,9cyclictest0-21swapper/220:37:332
2587799160,0cyclictest0-21swapper/219:50:062
25872991616,0cyclictest18339-21apt-get20:02:331
25872991614,1cyclictest611-21irqbalance19:59:391
25882991514,1cyclictest0-21swapper/320:55:013
2587799151,10cyclictest0-21swapper/220:52:342
25872991515,0cyclictest0-21swapper/120:42:341
25869991515,0cyclictest0-21swapper/019:42:330
2588299140,14cyclictest0-21swapper/320:40:253
2588299140,14cyclictest0-21swapper/319:50:403
2588299140,14cyclictest0-21swapper/319:11:473
25872991414,0cyclictest0-21swapper/119:11:471
2587299140,14cyclictest0-21swapper/119:20:221
2587299140,0cyclictest0-21swapper/120:50:111
2587299140,0cyclictest0-21swapper/120:10:261
2587299140,0cyclictest0-21swapper/119:40:041
25869991414,0cyclictest0-21swapper/020:41:070
2586999140,14cyclictest0-21swapper/019:55:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional