You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 10:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Jun 26, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10925992820,7cyclictest25290-21munin-plugin-st22:55:001
1092599270,26cyclictest13238-21ssh21:45:421
1092999260,25cyclictest930-21sh22:32:222
10925992623,3cyclictest0-21swapper/122:29:481
10925992623,3cyclictest0-21swapper/121:21:591
10925992620,5cyclictest0-21swapper/121:28:291
10925992619,6cyclictest32738-21ssh22:31:351
1092599260,25cyclictest934-21sh22:03:481
1092599260,25cyclictest0-21swapper/122:37:531
1092599260,25cyclictest0-21swapper/100:17:361
1092599260,21cyclictest0-21swapper/122:19:161
1092599260,21cyclictest0-21swapper/121:36:141
1092599260,1cyclictest0-21swapper/100:01:071
1093299250,24cyclictest34-21ksoftirqd/322:51:223
1093299250,24cyclictest34-21ksoftirqd/321:53:143
1093299250,24cyclictest331rcuc/300:25:393
10929992523,1cyclictest0-21swapper/223:00:282
10929992520,4cyclictest0-21swapper/222:17:012
1092999251,23cyclictest4175-21diskmemload22:25:122
1092999250,24cyclictest4175-21diskmemload22:00:112
10925992523,1cyclictest0-21swapper/100:24:161
10925992520,4cyclictest4175-21diskmemload23:26:561
10925992520,4cyclictest0-21swapper/122:20:501
10925992520,1cyclictest151rcu_preempt22:40:381
1092599250,24cyclictest4175-21diskmemload23:19:281
1092599250,24cyclictest0-21swapper/123:12:191
1092599250,24cyclictest0-21swapper/122:59:121
1092599250,24cyclictest0-21swapper/122:47:121
1092599250,24cyclictest0-21swapper/122:10:261
1092599250,0cyclictest0-21swapper/123:54:351
1092599250,0cyclictest0-21swapper/121:31:531
1092599250,0cyclictest0-21swapper/121:16:001
1092299250,24cyclictest4175-21diskmemload22:22:380
1092299250,24cyclictest0-21swapper/000:35:180
1092299250,23cyclictest0-21swapper/021:14:120
10932992422,1cyclictest0-21swapper/322:25:493
1093299240,23cyclictest34-21ksoftirqd/322:59:023
1093299240,23cyclictest0-21swapper/323:50:433
1093299240,1cyclictest4175-21diskmemload21:13:043
1093299240,1cyclictest28682-21grep21:30:423
1093299240,0cyclictest4175-21diskmemload00:13:003
10929992422,1cyclictest21089-21ssh22:50:432
10929992421,2cyclictest4568-21ssh21:37:552
1092999240,23cyclictest0-21swapper/223:40:572
1092999240,23cyclictest0-21swapper/222:56:032
1092999240,0cyclictest0-21swapper/223:53:432
10925992422,1cyclictest0-21swapper/123:07:091
10925992421,3cyclictest0-21swapper/121:54:231
10925992420,3cyclictest4175-21diskmemload23:39:301
10925992419,5cyclictest4175-21diskmemload23:23:451
1092599240,24cyclictest0-21swapper/100:12:571
1092599240,23cyclictest4175-21diskmemload23:49:321
1092599240,23cyclictest0-21swapper/100:05:201
1092599240,1cyclictest26770-21diskmemload21:57:291
1092599240,19cyclictest0-21swapper/123:02:341
1092599240,0cyclictest0-21swapper/123:33:351
1092599240,0cyclictest0-21swapper/122:06:581
1092599240,0cyclictest0-21swapper/121:41:151
1092599240,0cyclictest0-21swapper/100:39:551
1092299243,21cyclictest62550irq/127-eno123:52:120
10922992422,1cyclictest29356-21ssh22:00:380
10922992422,1cyclictest20241-21ssh22:50:300
10922992422,1cyclictest0-21swapper/023:08:390
1092299240,23cyclictest22328-21ssh00:17:270
1092299240,0cyclictest0-21swapper/023:16:000
1092299240,0cyclictest0-21swapper/022:29:050
1092299240,0cyclictest0-21swapper/022:17:260
1092299240,0cyclictest0-21swapper/000:05:020
10932992323,0cyclictest0-21swapper/323:43:023
10932992323,0cyclictest0-21swapper/322:08:183
10932992323,0cyclictest0-21swapper/321:45:363
10932992323,0cyclictest0-21swapper/319:55:373
10932992323,0cyclictest0-21swapper/300:30:283
10932992322,1cyclictest34-21ksoftirqd/322:24:013
10932992322,1cyclictest0-21swapper/321:16:423
1093299232,1cyclictest34-21ksoftirqd/322:45:373
10932992321,1cyclictest29550-21sh23:26:073
1093299230,23cyclictest31803-21kworker/u8:123:02:263
1093299230,22cyclictest4175-21diskmemload22:03:013
1093299230,22cyclictest4175-21diskmemload00:19:053
1093299230,22cyclictest34-21ksoftirqd/300:02:113
1093299230,22cyclictest19363-21ssh21:22:133
1093299230,1cyclictest4175-21diskmemload23:31:063
1093299230,0cyclictest0-21swapper/323:58:003
1093299230,0cyclictest0-21swapper/323:23:513
1093299230,0cyclictest0-21swapper/323:16:363
10929992323,0cyclictest0-21swapper/222:20:372
10929992321,2cyclictest0-21swapper/221:31:142
10929992321,1cyclictest0-21swapper/223:27:042
10929992321,1cyclictest0-21swapper/222:45:352
10929992321,1cyclictest0-21swapper/221:55:162
10929992321,1cyclictest0-21swapper/200:15:392
10929992321,1cyclictest0-21swapper/200:07:042
10929992320,2cyclictest31822-21awk21:00:012
10929992318,4cyclictest0-21swapper/223:46:102
1092999231,21cyclictest12768-1kworker/u9:122:42:042
1092999231,21cyclictest0-21swapper/223:05:212
1092999230,23cyclictest0-21swapper/221:45:132
1092999230,23cyclictest0-21swapper/200:36:052
1092999230,23cyclictest0-21swapper/200:31:272
1092999230,23cyclictest0-21swapper/200:01:462
1092999230,22cyclictest7125-21ssh22:09:472
1092999230,22cyclictest4175-21diskmemload23:55:382
1092999230,22cyclictest4175-21diskmemload23:32:352
1092999230,22cyclictest4175-21diskmemload21:27:412
1092999230,22cyclictest30534-21ssh00:25:122
1092999230,22cyclictest19260-21pluginstate21:50:452
1092999230,22cyclictest0-21swapper/221:41:282
1092999230,0cyclictest4175-21diskmemload22:14:582
1092999230,0cyclictest0-21swapper/222:38:012
1092999230,0cyclictest0-21swapper/221:11:362
1092999230,0cyclictest0-21swapper/200:20:402
10925992323,0cyclictest4175-21diskmemload23:56:551
10925992321,1cyclictest817-21apt-get19:55:371
10925992321,1cyclictest0-21swapper/121:10:101
10925992321,1cyclictest0-21swapper/120:40:001
10925992317,1cyclictest0-21swapper/123:41:131
1092599230,22cyclictest0-21swapper/120:30:381
10922992323,0cyclictest4175-21diskmemload21:46:030
10922992323,0cyclictest0-21swapper/023:36:300
10922992323,0cyclictest0-21swapper/022:40:580
10922992323,0cyclictest0-21swapper/000:30:100
10922992322,1cyclictest4175-21diskmemload23:11:510
10922992322,1cyclictest26651-21ssh21:29:330
10922992321,1cyclictest14300-21sh22:45:020
10922992321,1cyclictest0-21swapper/022:10:060
10922992321,1cyclictest0-21swapper/021:15:390
1092299230,23cyclictest0-21swapper/021:52:000
1092299230,22cyclictest4175-21diskmemload21:35:020
1092299230,22cyclictest4175-21diskmemload00:26:210
1092299230,22cyclictest0-21swapper/023:41:110
1092299230,22cyclictest0-21swapper/023:02:440
1092299230,22cyclictest0-21swapper/022:36:170
1092299230,21cyclictest0-21swapper/023:55:320
1092299230,21cyclictest0-21swapper/023:25:370
1092299230,21cyclictest0-21swapper/022:30:310
1092299230,1cyclictest18113-21munin-run19:25:010
1092299230,1cyclictest15216-21gzip00:10:470
1092299230,0cyclictest4175-21diskmemload21:44:420
1092299230,0cyclictest0-21swapper/023:30:050
1092299230,0cyclictest0-21swapper/000:24:160
10932992222,0cyclictest0-21swapper/300:37:313
10932992221,1cyclictest24957-21apt-get00:20:403
10932992220,1cyclictest9584-21diskstats22:40:383
10932992220,1cyclictest609-21rs:main3
10932992220,1cyclictest331rcuc/321:40:443
10932992220,1cyclictest3230-21sh22:35:123
10932992220,1cyclictest20547-21awk19:30:003
1093299221,21cyclictest34-21ksoftirqd/323:10:173
1093299220,21cyclictest4928-21rm23:05:193
1093299220,21cyclictest4175-21diskmemload21:55:343
1093299220,21cyclictest17333-21ssh23:45:023
1093299220,21cyclictest13444-21rm22:15:283
1093299220,1cyclictest7781-21sh00:05:083
1093299220,1cyclictest1813-21munin-run21:05:013
1093299220,1cyclictest1334-21sh22:32:593
10929992221,0cyclictest0-21swapper/223:35:462
10929992219,2cyclictest29391-21cron20:55:012
10929992218,3cyclictest32764-21hddtemp_smartct21:00:402
1092999220,21cyclictest0-21swapper/223:10:562
1092999220,21cyclictest0-21swapper/221:24:582
1092999220,21cyclictest0-21swapper/221:15:122
1092999220,21cyclictest0-21swapper/200:10:412
1092999220,1cyclictest22220-21idleruntime-cro20:40:002
1092999220,1cyclictest0-21swapper/223:16:542
10925992220,1cyclictest0-21swapper/120:10:371
1092599220,21cyclictest0-21swapper/121:00:381
1092599220,1cyclictest16003-21apt-get19:20:371
1092599220,1cyclictest0-21swapper/100:25:021
10922992222,0cyclictest0-21swapper/023:45:210
10922992220,2cyclictest0-21swapper/021:34:050
10922992220,1cyclictest4206-21sh00:00:560
10922992220,1cyclictest24383-21hddtemp_smartct21:55:410
10922992220,1cyclictest0-21swapper/022:05:030
1092299220,22cyclictest29681-21ssh22:57:520
1092299220,21cyclictest0-21swapper/020:15:380
1092299220,21cyclictest0-21swapper/019:30:370
1092299220,1cyclictest14966-21munin-run20:25:000
1092299220,1cyclictest0-21swapper/021:20:200
1092299220,0cyclictest0-21swapper/019:50:020
10932992120,1cyclictest867-21ssh21:35:113
1093299210,21cyclictest4175-21diskmemload22:10:143
1093299210,20cyclictest15253-21apt-get20:25:373
1093299210,1cyclictest0-21swapper/319:45:003
1093299210,0cyclictest331rcuc/320:15:013
1093299210,0cyclictest2103-21apt-get21:05:393
1093299210,0cyclictest10431-21apt-get20:15:383
1093299210,0cyclictest0-21swapper/319:45:363
10929992119,1cyclictest622-21gdbus19:45:002
10929992119,1cyclictest10104-21cron20:15:012
1092999210,20cyclictest0-21swapper/220:20:372
1092999210,0cyclictest0-21swapper/220:45:382
1092999210,0cyclictest0-21swapper/220:25:372
10925992121,0cyclictest0-21swapper/119:50:021
10925992120,0cyclictest0-21swapper/120:00:371
1092599210,20cyclictest0-21swapper/119:30:371
1092599210,0cyclictest0-21swapper/121:05:391
10922992121,0cyclictest0-21swapper/023:20:380
10922992121,0cyclictest0-21swapper/020:25:370
10922992120,1cyclictest3249-21apt-get20:00:370
10932992020,0cyclictest0-21swapper/320:40:363
10932992018,1cyclictest22218-21wc20:40:003
10929992020,0cyclictest0-21swapper/220:00:372
10929992020,0cyclictest0-21swapper/219:35:382
1092999200,0cyclictest0-21swapper/219:10:372
10925992020,0cyclictest0-21swapper/120:25:371
10925992018,1cyclictest31819-21munin-run21:00:011
1092599200,0cyclictest0-21swapper/119:10:371
10922992019,1cyclictest0-21swapper/019:45:360
10922992018,1cyclictest32115-21apt-get21:00:380
1092299200,0cyclictest0-21swapper/020:55:360
1092299200,0cyclictest0-21swapper/020:05:380
1092299200,0cyclictest0-21swapper/019:55:370
10932991919,0cyclictest0-21swapper/319:50:373
10932991919,0cyclictest0-21swapper/319:20:373
1093299190,1cyclictest34-21ksoftirqd/319:35:003
10929991919,0cyclictest0-21swapper/221:05:392
10929991919,0cyclictest0-21swapper/220:30:382
10929991919,0cyclictest0-21swapper/219:55:372
10925991919,0cyclictest14958-21perf20:25:001
10925991918,0cyclictest22513-21apt-get20:40:361
10922991919,0cyclictest23251-21apt-get19:35:390
10922991919,0cyclictest0-21swapper/019:10:370
1093299180,17cyclictest34-21ksoftirqd/320:45:383
10922991816,1cyclictest606-21dbus-daemon20:35:000
10932991717,0cyclictest0-21swapper/320:55:363
10932991716,1cyclictest11188-21apt-get19:10:373
10929991716,0cyclictest0-21swapper/219:50:382
10929991715,1cyclictest15708-21latency_hist19:20:012
10922991715,1cyclictest22202-21turbostat20:40:000
10929991616,0cyclictest0-21swapper/220:05:382
10929991615,1cyclictest0-21swapper/219:20:442
10929991615,0cyclictest20848-21apt-get19:30:372
10929991614,1cyclictest0-21swapper/219:25:432
1092999160,7cyclictest0-21swapper/220:40:362
1092599162,13cyclictest783-21snmpd20:17:011
10925991614,1cyclictest0-21swapper/120:55:011
10925991614,1cyclictest0-21swapper/120:45:481
10922991616,0cyclictest0-21swapper/021:05:390
10922991616,0cyclictest0-21swapper/020:40:360
10922991615,1cyclictest10140-21perf20:15:010
10922991614,1cyclictest0-21swapper/019:30:000
1092299160,15cyclictest26997-21latency_hist20:50:010
1092299160,10cyclictest0-21swapper/019:15:370
10929991512,2cyclictest11490-21ntp_states20:15:432
10925991513,1cyclictest30166-21perf19:50:001
1092299150,1cyclictest0-21swapper/020:50:400
10932991414,0cyclictest0-21swapper/320:08:103
10932991414,0cyclictest0-21swapper/320:00:373
10932991414,0cyclictest0-21swapper/319:35:383
1093299140,14cyclictest0-21swapper/320:51:423
1093299140,14cyclictest0-21swapper/320:30:073
1093299140,0cyclictest0-21swapper/319:15:193
1092999140,0cyclictest0-21swapper/219:45:042
10925991414,0cyclictest0-21swapper/120:05:091
10925991414,0cyclictest0-21swapper/119:35:021
1092599140,14cyclictest0-21swapper/119:25:061
10922991414,0cyclictest0-21swapper/019:40:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional