You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 06:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Tue Mar 17, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2623699270,25cyclictest313-21systemd-journal21:40:010
2623999260,25cyclictest3662-21sh23:28:393
2623799260,25cyclictest16490-21ssh23:41:171
2623999250,2cyclictest21966-21sh22:13:273
2623999250,2cyclictest21966-21sh22:13:273
2623999250,24cyclictest19901-21diskmemload00:25:143
2623999250,1cyclictest0-21swapper/322:06:253
26238992523,1cyclictest19365-21rm23:43:072
2623799250,24cyclictest19901-21diskmemload00:13:231
2623699250,0cyclictest0-21swapper/000:17:140
2623999240,24cyclictest0-21swapper/322:19:343
2623999240,23cyclictest0-21swapper/321:10:303
26238992422,1cyclictest0-21swapper/223:54:192
26238992419,5cyclictest22212-21ssh22:44:452
2623899240,23cyclictest0-21swapper/221:35:072
2623899240,23cyclictest0-21swapper/200:26:232
2623899240,1cyclictest26480-21ssh23:19:562
2623899240,0cyclictest0-21swapper/222:16:332
2623899240,0cyclictest0-21swapper/200:33:162
26237992423,1cyclictest0-21swapper/123:35:001
26237992422,1cyclictest31949-21sh23:25:001
26237992422,1cyclictest0-21swapper/123:24:001
26237992422,1cyclictest0-21swapper/121:45:001
2623799240,23cyclictest1143-21irqstats23:57:141
2623799240,23cyclictest0-21swapper/122:10:281
2623799240,23cyclictest0-21swapper/122:10:281
2623799240,23cyclictest0-21swapper/100:21:101
2623799240,1cyclictest20888-21ssh21:11:051
2623799240,0cyclictest0-21swapper/121:30:121
26236992423,1cyclictest19901-21diskmemload23:16:480
26236992423,1cyclictest0-21swapper/000:04:540
26236992421,2cyclictest25999-21ssh22:47:330
2623699240,24cyclictest0-21swapper/023:35:120
2623699240,24cyclictest0-21swapper/022:41:360
2623699240,23cyclictest3621-21ssh23:59:400
2623699240,23cyclictest19901-21diskmemload22:50:390
2623699240,23cyclictest0-21swapper/023:33:250
26239992323,0cyclictest0-21swapper/323:51:273
26239992323,0cyclictest0-21swapper/323:45:323
26239992323,0cyclictest0-21swapper/300:01:443
26239992322,1cyclictest19901-21diskmemload23:00:343
26239992322,1cyclictest19901-21diskmemload00:13:083
26239992322,0cyclictest19901-21diskmemload22:31:333
2623999230,2cyclictest19901-21diskmemload00:09:433
2623999230,23cyclictest0-21swapper/323:55:163
2623999230,23cyclictest0-21swapper/321:57:553
2623999230,23cyclictest0-21swapper/321:54:183
2623999230,23cyclictest0-21swapper/321:25:053
2623999230,22cyclictest8648-21ssh00:35:263
2623999230,22cyclictest19901-21diskmemload23:37:163
2623999230,22cyclictest19901-21diskmemload21:22:123
2623999230,22cyclictest19901-21diskmemload21:16:293
2623999230,22cyclictest18852-21cron21:40:013
2623999230,22cyclictest15636-21sh22:37:423
2623999230,22cyclictest14840-21ssh23:08:033
2623999230,22cyclictest0-21swapper/322:45:283
2623999230,22cyclictest0-21swapper/322:27:253
2623999230,1cyclictest21656-21ssh22:43:563
2623999230,1cyclictest10100-21ssh21:32:113
2623999230,1cyclictest0-21swapper/321:00:003
2623999230,1cyclictest0-21swapper/300:30:383
2623999230,0cyclictest0-21swapper/323:10:463
26238992323,0cyclictest0-21swapper/221:13:412
26238992321,1cyclictest0-21swapper/220:15:002
26238992320,2cyclictest16603-21ssh22:39:092
2623899230,2cyclictest0-21swapper/221:53:292
2623899230,23cyclictest19901-21diskmemload00:35:182
2623899230,23cyclictest0-21swapper/223:37:092
2623899230,23cyclictest0-21swapper/223:27:282
2623899230,22cyclictest7394-21unixbench_singl00:02:212
2623899230,22cyclictest5644-21sh23:00:012
2623899230,22cyclictest25642-21sh23:49:432
2623899230,22cyclictest25556-21ssh21:46:532
2623899230,22cyclictest2297-21ssh23:57:322
2623899230,22cyclictest22768-21rm22:45:122
2623899230,22cyclictest17886-21turbostat.cron20:00:012
2623899230,22cyclictest16653-21ssh22:08:112
2623899230,22cyclictest11457-21ssh23:05:282
2623899230,22cyclictest0-21swapper/221:17:242
2623899230,21cyclictest0-21swapper/220:50:012
2623899230,1cyclictest0-21swapper/223:30:532
2623899230,0cyclictest19901-21diskmemload22:13:022
2623899230,0cyclictest19901-21diskmemload22:13:022
2623899230,0cyclictest0-21swapper/200:13:072
26237992323,0cyclictest19901-21diskmemload22:55:581
26237992323,0cyclictest18306-21apt-get20:02:111
26237992323,0cyclictest0-21swapper/123:37:021
26237992323,0cyclictest0-21swapper/123:07:171
26237992323,0cyclictest0-21swapper/122:36:291
26237992323,0cyclictest0-21swapper/121:36:371
26237992321,1cyclictest4341-21ssh21:26:271
26237992320,2cyclictest5854-21cstates00:02:131
26237992320,2cyclictest27501-21sendmail-msp20:20:001
2623799230,23cyclictest0-21swapper/122:21:011
2623799230,23cyclictest0-21swapper/122:03:531
2623799230,22cyclictest7101-21sh21:59:351
2623799230,22cyclictest24423-21ssh22:16:121
2623799230,22cyclictest20550-21rm00:16:091
2623799230,22cyclictest17712-21ssh22:40:341
2623799230,22cyclictest1550-21rm00:28:001
2623799230,22cyclictest0-21swapper/123:45:561
2623799230,1cyclictest0-21swapper/119:35:001
2623799230,0cyclictest19901-21diskmemload22:31:091
2623799230,0cyclictest0-21swapper/100:05:491
26236992323,0cyclictest62550irq/127-eno123:49:270
26236992323,0cyclictest19901-21diskmemload21:30:550
26236992323,0cyclictest0-21swapper/023:07:210
26236992323,0cyclictest0-21swapper/000:35:230
26236992323,0cyclictest0-21swapper/000:23:230
26236992322,1cyclictest62550irq/127-eno123:44:390
2623699232,21cyclictest62550irq/127-eno100:13:250
26236992321,1cyclictest0-21swapper/022:07:050
26236992321,1cyclictest0-21swapper/021:13:150
26236992321,1cyclictest0-21swapper/020:07:130
2623699230,23cyclictest0-21swapper/022:23:500
2623699230,23cyclictest0-21swapper/021:52:050
2623699230,23cyclictest0-21swapper/021:20:020
2623699230,22cyclictest19901-21diskmemload21:40:240
2623699230,22cyclictest0-21swapper/022:58:590
2623699230,21cyclictest0-21swapper/022:36:340
2623699230,0cyclictest0-21swapper/023:21:020
2623699230,0cyclictest0-21swapper/022:30:190
2623699230,0cyclictest0-21swapper/021:48:570
26239992222,0cyclictest0-21swapper/323:33:243
26239992220,1cyclictest0-21swapper/322:22:033
26239992218,0cyclictest0-21swapper/321:47:573
2623999220,2cyclictest23896-21sort23:17:153
2623999220,21cyclictest3838-21apt-get19:32:113
2623999220,21cyclictest19901-21diskmemload22:00:033
2623999220,21cyclictest19901-21diskmemload00:20:373
2623999220,21cyclictest0-21swapper/322:56:183
2623999220,21cyclictest0-21swapper/322:50:433
2623999220,21cyclictest0-21swapper/320:37:123
2623999220,21cyclictest0-21swapper/320:07:133
2623999220,1cyclictest31112-21ssh23:24:003
2623999220,1cyclictest0-21swapper/323:40:253
2623999220,0cyclictest0-21swapper/321:43:123
26238992222,0cyclictest0-21swapper/222:20:482
26238992222,0cyclictest0-21swapper/221:40:022
26238992222,0cyclictest0-21swapper/221:21:012
26238992220,1cyclictest10425-21ssh00:06:312
2623899220,22cyclictest19901-21diskmemload22:52:082
2623899220,22cyclictest19901-21diskmemload21:26:062
2623899220,22cyclictest19901-21diskmemload00:21:062
2623899220,22cyclictest0-21swapper/223:10:302
2623899220,21cyclictest3418-21perf19:30:002
2623899220,21cyclictest28603-21dump-pmu-power19:15:002
2623899220,21cyclictest0-21swapper/221:07:112
2623899220,21cyclictest0-21swapper/220:27:122
2623899220,1cyclictest8465-21rm22:01:202
2623899220,1cyclictest0-21swapper/222:26:342
2623899220,0cyclictest0-21swapper/223:21:062
2623899220,0cyclictest0-21swapper/222:30:062
26237992220,1cyclictest32752-21ssh22:55:001
26237992220,1cyclictest29890-21apt-get20:25:021
26237992220,1cyclictest0-21swapper/122:45:261
2623799220,2cyclictest26883-21ntp_states21:47:161
2623799220,22cyclictest0-21swapper/121:51:511
2623799220,22cyclictest0-21swapper/100:32:521
2623799220,21cyclictest27105-21diskmemload23:51:311
2623799220,21cyclictest2359-21sh22:26:001
2623799220,21cyclictest17895-21awk20:00:011
2623799220,21cyclictest0-21swapper/119:52:111
2623799220,21cyclictest0-21swapper/100:35:171
2623799220,1cyclictest22632-21rm23:16:441
2623799220,1cyclictest0-21swapper/122:05:071
26236992222,0cyclictest0-21swapper/023:10:010
26236992222,0cyclictest0-21swapper/000:06:290
26236992221,1cyclictest19901-21diskmemload21:25:250
26236992220,1cyclictest0-21swapper/023:51:110
26236992220,1cyclictest0-21swapper/020:15:000
2623699220,22cyclictest19901-21diskmemload22:19:280
2623699220,22cyclictest19901-21diskmemload22:10:370
2623699220,22cyclictest19901-21diskmemload22:10:370
2623699220,21cyclictest30998-21sh19:20:000
2623699220,21cyclictest19901-21diskmemload23:00:110
2623699220,21cyclictest0-21swapper/022:00:020
2623699220,21cyclictest0-21swapper/021:55:520
2623699220,21cyclictest0-21swapper/019:12:110
2623699220,1cyclictest19933-21munin-run21:10:000
2623699220,1cyclictest19901-21diskmemload21:17:030
2623699220,1cyclictest0-21swapper/023:27:210
2623699220,1cyclictest0-21swapper/000:25:180
2623699220,0cyclictest19901-21diskmemload00:30:430
2623999210,21cyclictest19901-21diskmemload00:17:573
2623999210,1cyclictest0-21swapper/319:15:003
2623999210,0cyclictest0-21swapper/319:27:123
26238992121,0cyclictest19901-21diskmemload23:00:232
26238992120,1cyclictest25493-21apt-get20:17:102
26238992119,1cyclictest0-21swapper/221:30:042
2623899210,20cyclictest0-21swapper/221:02:102
2623899210,0cyclictest0-21swapper/219:22:122
26237992121,0cyclictest19901-21diskmemload23:00:331
26237992121,0cyclictest0-21swapper/121:20:111
26237992121,0cyclictest0-21swapper/119:45:011
2623799210,20cyclictest15554-21apt-get21:02:101
2623799210,0cyclictest0-21swapper/121:15:451
26236992121,0cyclictest0-21swapper/021:02:090
26236992121,0cyclictest0-21swapper/019:37:110
26236992121,0cyclictest0-21swapper/019:22:120
26236992119,1cyclictest5823-21apt19:35:000
26236992119,1cyclictest3416-21latency_hist19:30:000
2623699210,1cyclictest0-21swapper/019:50:010
2623699210,0cyclictest13494-21apt-get19:52:110
26239992020,0cyclictest0-21swapper/319:52:113
26239992019,1cyclictest0-21swapper/320:32:123
2623999200,0cyclictest5506-21apt-get20:42:123
26238992020,0cyclictest0-21swapper/220:37:122
26238992018,1cyclictest0-21swapper/220:02:112
2623899200,0cyclictest6235-21apt-get19:37:112
26236992020,0cyclictest0-21swapper/020:02:110
2623699200,0cyclictest0-21swapper/020:32:120
26239991919,0cyclictest0-21swapper/319:37:113
26239991919,0cyclictest0-21swapper/319:22:123
26238991918,1cyclictest13139-21apt-get20:57:112
26237991919,0cyclictest1429-21apt-get19:27:121
26237991919,0cyclictest0-21swapper/120:57:121
26237991919,0cyclictest0-21swapper/120:52:111
26236991919,0cyclictest0-21swapper/020:37:120
26236991918,1cyclictest0-21swapper/020:42:120
26236991917,1cyclictest17859-21sh20:00:010
26239991818,0cyclictest0-21swapper/320:12:123
26239991817,1cyclictest9928-21munin-run20:50:013
26237991817,1cyclictest31459-21apt-get19:22:121
2623799180,17cyclictest27917-21apt-get20:22:101
26236991818,0cyclictest0-21swapper/020:52:110
26236991817,0cyclictest0-21swapper/020:57:120
26239991717,0cyclictest29890-21apt-get20:25:023
26239991717,0cyclictest0-21swapper/321:07:103
26239991716,1cyclictest10670-21latency_hist19:45:013
26239991715,1cyclictest27446-21cron20:20:003
26238991717,0cyclictest0-21swapper/220:22:102
26237991717,0cyclictest0-21swapper/120:42:121
26237991717,0cyclictest0-21swapper/120:32:121
26237991717,0cyclictest0-21swapper/120:07:131
26237991717,0cyclictest0-21swapper/119:37:111
26237991716,1cyclictest31027-21turbostat19:20:001
26237991715,1cyclictest0-21swapper/121:10:001
26236991717,0cyclictest0-21swapper/020:47:110
26236991717,0cyclictest0-21swapper/020:25:020
26236991716,0cyclictest0-21swapper/019:42:120
26239991616,0cyclictest10347-21apt-get20:52:113
2623999161,14cyclictest0-21swapper/320:02:133
26238991615,1cyclictest0-21swapper/219:20:002
26238991614,1cyclictest313-21systemd-journal19:55:012
26237991616,0cyclictest0-21swapper/120:47:111
26236991616,0cyclictest0-21swapper/020:22:100
26239991513,1cyclictest29897-21latency_hist20:25:003
26238991515,0cyclictest20702-21apt-get20:07:132
26238991515,0cyclictest0-21swapper/220:42:122
26238991515,0cyclictest0-21swapper/219:47:122
2623999140,14cyclictest0-21swapper/321:00:113
2623999140,14cyclictest0-21swapper/319:45:063
2623999140,14cyclictest0-21swapper/319:15:323
2623999140,0cyclictest0-21swapper/319:55:093
2623899140,14cyclictest0-21swapper/220:32:232
2623899140,14cyclictest0-21swapper/219:30:062
2623899140,0cyclictest0-21swapper/220:50:292
2623899140,0cyclictest0-21swapper/219:40:402
26237991414,0cyclictest0-21swapper/120:10:021
2623799140,0cyclictest0-21swapper/119:11:281
2623699140,0cyclictest0-21swapper/020:15:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional