You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 05:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jul 21, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
822299300,5cyclictest26424-21sh22:00:092
822799270,26cyclictest0-21swapper/322:18:233
8222992720,5cyclictest32083-21ssh22:33:532
822299270,26cyclictest2642-21ssh21:38:342
822299270,26cyclictest12976-21perf19:20:002
822299270,26cyclictest0-21swapper/222:52:422
822299270,23cyclictest29866-21ssh21:34:082
822299270,1cyclictest1856-21diskmemload23:05:592
822299270,0cyclictest0-21swapper/222:22:502
821999271,1cyclictest0-21swapper/122:02:521
821999270,1cyclictest15942-21munin-run22:20:001
8222992621,4cyclictest0-21swapper/223:49:442
822299260,25cyclictest9274-21rm21:16:542
822299260,25cyclictest7465-21ssh21:14:382
822299260,25cyclictest1998-21fschecks_count22:07:432
822299260,25cyclictest0-21swapper/222:20:002
822299260,23cyclictest1856-21diskmemload21:21:282
822299260,1cyclictest1856-21diskmemload23:02:512
8219992623,3cyclictest1856-21diskmemload00:29:061
821999260,22cyclictest0-21swapper/100:17:511
821999260,21cyclictest0-21swapper/121:22:511
821999260,0cyclictest0-21swapper/100:30:421
8217992624,1cyclictest17792-21munin-run19:30:010
822799250,24cyclictest5936-21ssh22:11:473
822799250,24cyclictest1856-21diskmemload23:05:363
822799250,24cyclictest13257-21perf21:20:013
8222992520,4cyclictest1856-21diskmemload00:15:192
822299250,4cyclictest0-21swapper/223:42:072
822299250,25cyclictest0-21swapper/200:03:112
822299250,24cyclictest32353-21apt-get21:07:422
822299250,24cyclictest26986-21ssh23:55:532
822299250,24cyclictest1856-21diskmemload23:18:382
822299250,24cyclictest13097-21apt-get20:27:412
822299250,24cyclictest0-21swapper/219:52:422
822299250,24cyclictest0-21swapper/200:14:012
8219992521,3cyclictest1856-21diskmemload21:35:001
8219992521,3cyclictest0-21swapper/123:01:391
821999250,24cyclictest0-21swapper/123:47:551
821999250,24cyclictest0-21swapper/122:13:441
821999250,24cyclictest0-21swapper/121:10:511
821999250,21cyclictest22220-21ssh21:57:011
821999250,1cyclictest3296-21grep22:07:501
821999250,1cyclictest0-21swapper/100:35:471
8217992523,1cyclictest22805-21sh22:26:170
821799250,0cyclictest0-21swapper/023:19:320
8227992420,3cyclictest16317-21cut00:15:013
822799240,23cyclictest27932-21basename00:25:013
822799240,1cyclictest31628-21ssh22:04:523
8222992422,2cyclictest0-21swapper/200:35:292
8222992421,3cyclictest1856-21diskmemload21:27:232
8222992420,3cyclictest0-21swapper/221:47:432
8222992419,4cyclictest32540-21sh23:32:342
8222992419,4cyclictest25417-21sh23:54:052
822299240,4cyclictest0-21swapper/200:26:472
822299240,3cyclictest1856-21diskmemload21:55:292
822299240,23cyclictest15845-21ssh21:50:592
822299240,23cyclictest15845-21ssh21:50:592
822299240,23cyclictest12470-21ssh22:45:412
822299240,23cyclictest0-21swapper/223:13:362
822299240,23cyclictest0-21swapper/222:39:032
822299240,0cyclictest1856-21diskmemload22:40:032
822299240,0cyclictest1856-21diskmemload22:13:462
822299240,0cyclictest0-21swapper/223:23:102
8219992422,2cyclictest0-21swapper/121:43:391
8219992421,2cyclictest2643-21grep20:05:001
821999240,2cyclictest29048-21cat23:57:441
821999240,24cyclictest0-21swapper/122:22:211
821999240,23cyclictest0-21swapper/122:37:021
821999240,21cyclictest0-21swapper/123:10:341
821999240,1cyclictest19960-21ssh23:20:341
821999240,0cyclictest1856-21diskmemload00:09:081
821999240,0cyclictest0-21swapper/123:54:051
821999240,0cyclictest0-21swapper/121:15:381
821999240,0cyclictest0-21swapper/119:47:411
8217992421,2cyclictest0-21swapper/023:31:000
821799240,24cyclictest0-21swapper/023:14:450
821799240,24cyclictest0-21swapper/000:27:430
821799240,23cyclictest29497-21munin-plugin-st21:00:000
821799240,23cyclictest10240-21ls20:20:010
821799240,23cyclictest0-21swapper/021:45:170
821799240,23cyclictest0-21swapper/000:05:300
821799240,1cyclictest0-21swapper/023:07:050
821799240,0cyclictest1856-21diskmemload21:11:030
8227992323,0cyclictest1856-21diskmemload23:46:163
8227992323,0cyclictest1856-21diskmemload23:30:093
8227992323,0cyclictest1856-21diskmemload00:32:433
8227992323,0cyclictest0-21swapper/323:42:203
8227992323,0cyclictest0-21swapper/323:20:333
8227992322,1cyclictest0-21swapper/322:49:003
8227992321,1cyclictest8633-21apt-get19:12:403
8227992321,1cyclictest0-21swapper/300:05:363
822799230,2cyclictest4667-21ssh21:40:583
822799230,2cyclictest31069-21ssh21:35:293
822799230,23cyclictest22472-21sh21:57:213
822799230,23cyclictest1856-21diskmemload00:17:403
822799230,23cyclictest0-21swapper/323:51:083
822799230,23cyclictest0-21swapper/322:52:093
822799230,23cyclictest0-21swapper/321:27:483
822799230,23cyclictest0-21swapper/321:11:003
822799230,23cyclictest0-21swapper/300:01:333
822799230,22cyclictest8621-21ssh00:37:233
822799230,22cyclictest27825-21ssh23:57:083
822799230,22cyclictest15490-21apt-get20:32:403
822799230,1cyclictest866-21ssh22:35:063
822799230,1cyclictest28497-21latency_hist23:00:003
822799230,1cyclictest28392-21ssh23:27:493
822799230,0cyclictest1856-21diskmemload23:16:163
822799230,0cyclictest1856-21diskmemload23:04:573
822799230,0cyclictest0-21swapper/322:20:303
822799230,0cyclictest0-21swapper/322:05:143
8222992323,0cyclictest25844-21apt-get19:47:412
8222992323,0cyclictest0-21swapper/223:35:162
8222992320,2cyclictest313-21systemd-journal20:45:002
8222992320,2cyclictest23676-21ssh00:21:332
822299230,22cyclictest27765-21/usr/sbin/munin23:27:462
822299230,21cyclictest0-21swapper/220:10:002
822299230,20cyclictest0-21swapper/200:34:042
822299230,0cyclictest0-21swapper/222:55:312
822299230,0cyclictest0-21swapper/200:06:122
8219992321,2cyclictest0-21swapper/100:21:361
8219992320,3cyclictest0-21swapper/122:40:231
8219992320,3cyclictest0-21swapper/122:29:231
8219992320,3cyclictest0-21swapper/121:50:461
8219992320,3cyclictest0-21swapper/121:50:461
8219992320,3cyclictest0-21swapper/121:27:471
8219992320,2cyclictest0-21swapper/123:15:141
8219992319,3cyclictest0-21swapper/123:09:181
821999230,23cyclictest0-21swapper/123:32:341
821999230,23cyclictest0-21swapper/122:47:571
821999230,23cyclictest0-21swapper/121:37:141
821999230,23cyclictest0-21swapper/121:02:411
821999230,22cyclictest7813-21dump-pmu-power20:15:011
821999230,22cyclictest22244-21munin-run20:45:001
821999230,22cyclictest1856-21diskmemload22:57:401
821999230,22cyclictest17586-21sh22:50:021
821999230,22cyclictest0-21swapper/100:00:141
821999230,0cyclictest0-21swapper/123:25:201
821999230,0cyclictest0-21swapper/121:47:481
8217992323,0cyclictest0-21swapper/000:35:140
8217992322,1cyclictest62550irq/127-eno122:15:220
8217992321,1cyclictest1856-21diskmemload23:55:510
8217992321,1cyclictest0-21swapper/023:01:130
8217992321,1cyclictest0-21swapper/022:06:430
8217992321,1cyclictest0-21swapper/000:31:480
821799230,23cyclictest1856-21diskmemload21:23:580
821799230,23cyclictest0-21swapper/023:29:090
821799230,23cyclictest0-21swapper/021:55:240
821799230,23cyclictest0-21swapper/021:51:080
821799230,23cyclictest0-21swapper/021:51:080
821799230,23cyclictest0-21swapper/021:30:530
821799230,23cyclictest0-21swapper/021:29:150
821799230,23cyclictest0-21swapper/000:17:030
821799230,22cyclictest5393-21sh22:10:170
821799230,22cyclictest23887-21rm00:21:490
821799230,22cyclictest1856-21diskmemload22:30:380
821799230,22cyclictest0-21swapper/023:20:120
821799230,22cyclictest0-21swapper/022:43:570
821799230,22cyclictest0-21swapper/019:40:020
821799230,1cyclictest0-21swapper/022:04:250
821799230,1cyclictest0-21swapper/021:42:550
821799230,0cyclictest1856-21diskmemload00:00:160
821799230,0cyclictest0-21swapper/022:55:020
821799230,0cyclictest0-21swapper/021:36:360
8227992222,0cyclictest1856-21diskmemload00:25:033
8227992222,0cyclictest0-21swapper/323:36:473
8227992222,0cyclictest0-21swapper/322:40:293
8227992221,1cyclictest1856-21diskmemload22:27:433
8227992220,1cyclictest25386-21basename19:45:003
822799220,22cyclictest0-21swapper/321:45:063
822799220,21cyclictest28294-21apt-get19:52:423
822799220,21cyclictest15575-21sh21:50:383
822799220,21cyclictest15575-21sh21:50:383
822799220,21cyclictest12637-21basename20:25:013
822799220,21cyclictest0-21swapper/321:30:073
822799220,21cyclictest0-21swapper/321:21:243
822799220,1cyclictest8501-21rm23:10:193
822799220,1cyclictest27006-21cron20:55:003
822799220,1cyclictest1856-21diskmemload22:30:133
8222992220,1cyclictest7817-21turbostat20:15:012
8222992220,1cyclictest4744-21ssh21:41:032
8222992220,1cyclictest10567-21apt19:15:002
8222992219,2cyclictest17788-21awk19:30:012
822299220,21cyclictest0-21swapper/220:57:422
8219992220,2cyclictest0-21swapper/123:39:531
8219992220,1cyclictest0-21swapper/122:30:101
821999220,1cyclictest10565-21perf19:15:001
8217992222,0cyclictest0-21swapper/022:50:360
8217992222,0cyclictest0-21swapper/021:15:480
8217992220,1cyclictest20169-21latency_hist19:35:010
8217992220,1cyclictest0-21swapper/023:50:140
8217992220,1cyclictest0-21swapper/022:37:080
821799220,2cyclictest11318-21sh00:10:310
821799220,21cyclictest1856-21diskmemload23:35:200
821799220,21cyclictest15390-21apt19:25:010
821799220,21cyclictest0-21swapper/022:45:310
821799220,21cyclictest0-21swapper/020:27:410
821799220,1cyclictest0-21swapper/023:40:490
821799220,1cyclictest0-21swapper/022:20:440
821799220,1cyclictest0-21swapper/020:40:000
821799220,0cyclictest0-21swapper/023:45:360
822799210,0cyclictest0-21swapper/319:57:413
8222992121,0cyclictest0-21swapper/221:02:412
822299210,1cyclictest22587-21perf19:40:012
8219992118,2cyclictest24617-21sh20:50:001
821999210,20cyclictest0-21swapper/120:05:021
821999210,1cyclictest8930-21sh20:17:451
821999210,0cyclictest0-21swapper/120:57:421
821799210,1cyclictest2625-21apt-get20:05:020
821799210,1cyclictest0-21swapper/019:17:420
821799210,0cyclictest0-21swapper/020:12:410
821799210,0cyclictest0-21swapper/019:12:410
8227992020,0cyclictest0-21swapper/321:07:423
8227992020,0cyclictest0-21swapper/320:12:403
8227992019,1cyclictest8285-21apt-get20:17:423
8227992019,1cyclictest0-21swapper/320:27:413
8227992018,1cyclictest15392-21sleep19:25:013
8222992018,1cyclictest2641-21basename20:05:002
822299200,0cyclictest0-21swapper/220:37:432
822299200,0cyclictest0-21swapper/220:17:422
821999200,1cyclictest0-21swapper/119:37:421
821999200,19cyclictest30686-21apt-get19:57:411
8217992020,0cyclictest0-21swapper/019:57:410
8217992019,1cyclictest0-21swapper/019:52:420
8217992018,1cyclictest24627-21grep20:50:000
821799200,0cyclictest27784-21perf19:50:000
821799200,0cyclictest25093-21apt-get20:52:420
8227991919,0cyclictest0-21swapper/319:27:433
8227991917,1cyclictest24620-21cron20:50:003
8227991917,1cyclictest0-21swapper/319:40:013
822799190,18cyclictest0-21swapper/321:02:413
822299190,18cyclictest13451-21apt-get19:22:412
822299190,18cyclictest0-21swapper/220:47:472
8219991919,0cyclictest0-21swapper/120:27:411
8219991919,0cyclictest0-21swapper/119:32:411
8219991918,1cyclictest17892-21apt-get20:37:431
821999191,17cyclictest0-21swapper/119:22:491
8222991817,0cyclictest0-21swapper/219:57:412
822299180,17cyclictest0-21swapper/220:52:422
822299180,0cyclictest0-21swapper/220:22:422
8219991818,0cyclictest0-21swapper/119:27:431
8219991816,1cyclictest1873-21perf21:10:011
821999180,1cyclictest783-21snmpd20:22:421
8217991816,1cyclictest0-21swapper/020:45:000
8222991716,0cyclictest0-21swapper/219:32:412
822299170,16cyclictest24501-21ntp_states19:42:472
821999170,1cyclictest26141-21ntp_states20:52:481
821999170,12cyclictest23452-21apt-get19:42:421
8217991717,0cyclictest0-21swapper/021:07:420
8227991614,1cyclictest0-21swapper/320:02:403
822799160,15cyclictest0-21swapper/320:57:523
8217991616,0cyclictest0-21swapper/020:32:390
8217991616,0cyclictest0-21swapper/020:22:410
8217991616,0cyclictest0-21swapper/019:37:410
8227991515,0cyclictest0-21swapper/320:37:433
822299150,14cyclictest0-21swapper/220:30:542
821999150,1cyclictest16790-21ptp4l-jitter20:32:461
8217991511,4cyclictest32717-21kworker/0:220:02:140
8227991414,0cyclictest0-21swapper/319:30:363
822799140,14cyclictest0-21swapper/320:41:353
822799140,14cyclictest0-21swapper/319:45:113
822799140,14cyclictest0-21swapper/319:15:323
822799140,0cyclictest0-21swapper/320:05:083
821999140,0cyclictest0-21swapper/119:15:391
821799140,14cyclictest0-21swapper/021:01:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional