You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 06:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Aug 22, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23329992623,2cyclictest9040-21/usr/sbin/munin22:00:202
2333099250,24cyclictest24807-21apt-get21:45:193
2333099250,24cyclictest16997-21diskmemload00:02:543
2333099250,24cyclictest0-21swapper/322:26:513
2333099250,24cyclictest0-21swapper/300:12:463
2332999250,24cyclictest0-21swapper/222:36:402
2332999250,1cyclictest0-21swapper/222:22:352
2332899250,24cyclictest31073-21ssh22:19:291
2332899250,24cyclictest16997-21diskmemload22:37:361
2332899250,24cyclictest16997-21diskmemload21:12:551
23327992523,1cyclictest16997-21diskmemload23:52:020
2332799250,24cyclictest0-21swapper/023:05:340
23330992423,1cyclictest0-21swapper/323:45:053
2333099240,23cyclictest0-21swapper/322:50:193
2333099240,23cyclictest0-21swapper/322:39:183
2333099240,23cyclictest0-21swapper/321:52:263
23329992422,1cyclictest0-21swapper/221:50:332
2332999240,23cyclictest16997-21diskmemload22:19:482
2332999240,23cyclictest0-21swapper/223:29:592
2332999240,23cyclictest0-21swapper/223:03:372
2332999240,23cyclictest0-21swapper/222:27:402
2332999240,23cyclictest0-21swapper/221:21:002
2332999240,23cyclictest0-21swapper/200:19:202
2332999240,1cyclictest0-21swapper/200:04:042
2332999240,0cyclictest0-21swapper/222:54:532
23328992422,1cyclictest28140-21grep19:20:011
2332899240,24cyclictest32522-21ssh23:18:131
2332899240,24cyclictest0-21swapper/123:07:211
2332899240,24cyclictest0-21swapper/122:12:341
2332899240,24cyclictest0-21swapper/121:30:141
2332899240,23cyclictest606-21dbus-daemon20:00:011
2332899240,23cyclictest11555-21idleruntime-cro23:00:001
2332899240,23cyclictest0-21swapper/123:30:111
2332899240,0cyclictest16997-21diskmemload23:01:171
23327992424,0cyclictest16997-21diskmemload22:56:230
23327992423,1cyclictest0-21swapper/022:12:300
23327992423,1cyclictest0-21swapper/000:02:480
23327992422,1cyclictest606-21dbus-daemon20:40:000
2332799240,24cyclictest0-21swapper/023:41:580
2332799240,24cyclictest0-21swapper/021:55:290
2332799240,23cyclictest10560-21sh23:27:080
2332799240,23cyclictest0-21swapper/022:23:340
23330992323,0cyclictest16997-21diskmemload22:17:063
23330992323,0cyclictest0-21swapper/323:35:483
23330992323,0cyclictest0-21swapper/323:30:473
23330992323,0cyclictest0-21swapper/323:19:473
23330992323,0cyclictest0-21swapper/321:44:383
23330992322,1cyclictest8886-21apt-get23:55:193
23330992321,1cyclictest30805-21apt-get19:25:173
23330992321,1cyclictest0-21swapper/322:41:203
23330992321,1cyclictest0-21swapper/322:30:203
2333099230,23cyclictest0-21swapper/321:32:253
2333099230,22cyclictest7023-21latency_hist23:25:013
2333099230,22cyclictest6314-21sh22:55:143
2333099230,22cyclictest29754-21ssh22:45:493
2333099230,22cyclictest24142-21sh00:08:223
2333099230,22cyclictest16997-21diskmemload21:10:233
2333099230,22cyclictest16997-21diskmemload00:35:373
2333099230,0cyclictest16997-21diskmemload23:02:033
23329992323,0cyclictest0-21swapper/223:05:582
23329992323,0cyclictest0-21swapper/222:05:352
23329992323,0cyclictest0-21swapper/200:30:302
23329992323,0cyclictest0-21swapper/200:11:222
23329992322,0cyclictest0-21swapper/223:42:312
23329992321,1cyclictest0-21swapper/223:35:062
23329992321,1cyclictest0-21swapper/222:43:302
23329992321,1cyclictest0-21swapper/200:29:442
2332999231,21cyclictest1438-21turbostat23:20:012
2332999231,21cyclictest11884-21sh23:57:112
2332999230,2cyclictest17390-21ssh21:38:172
2332999230,22cyclictest7036-21ssh00:20:492
2332999230,22cyclictest0-21swapper/221:12:002
2332999230,0cyclictest0-21swapper/223:33:502
2332999230,0cyclictest0-21swapper/221:55:222
2332999230,0cyclictest0-21swapper/221:25:592
2332999230,0cyclictest0-21swapper/220:00:192
23328992323,0cyclictest0-21swapper/122:41:181
23328992322,1cyclictest19074-21ssh21:40:121
23328992321,2cyclictest23553-21rm00:07:381
23328992321,1cyclictest32307-21fw_packets00:15:221
23328992321,1cyclictest0-21swapper/122:30:081
2332899230,2cyclictest27487-21ssh21:46:411
2332899230,2cyclictest11016-21ssh23:55:571
2332899230,23cyclictest16997-21diskmemload23:38:181
2332899230,23cyclictest0-21swapper/123:48:271
2332899230,23cyclictest0-21swapper/121:23:361
2332899230,23cyclictest0-21swapper/100:26:321
2332899230,22cyclictest7680-21sh00:21:471
2332899230,22cyclictest473-21grep22:50:001
2332899230,22cyclictest1997-21ssh23:20:221
2332899230,22cyclictest16997-21diskmemload00:00:011
2332899230,22cyclictest15968-21ssh00:30:051
2332899230,1cyclictest13550-21ssh21:35:231
2332899230,0cyclictest16997-21diskmemload21:17:081
2332899230,0cyclictest0-21swapper/121:52:201
23327992323,0cyclictest16997-21diskmemload23:22:160
23327992323,0cyclictest0-21swapper/000:15:310
23327992322,1cyclictest18679-21ssh22:36:030
23327992322,1cyclictest0-21swapper/021:29:100
23327992321,1cyclictest0-21swapper/020:05:190
2332799230,2cyclictest6087-21sh22:55:000
2332799230,2cyclictest11683-21ssh22:01:360
2332799230,23cyclictest12340-21ssh23:57:490
2332799230,23cyclictest0-21swapper/022:33:170
2332799230,23cyclictest0-21swapper/021:23:250
2332799230,23cyclictest0-21swapper/021:10:160
2332799230,22cyclictest5047-21sh22:25:170
2332799230,22cyclictest32171-21idleruntime-cro20:35:000
2332799230,22cyclictest15952-21ssh00:30:040
2332799230,22cyclictest13000-21grep21:35:000
2332799230,22cyclictest1027-21ssh23:47:390
2332799230,1cyclictest16997-21diskmemload23:17:180
2332799230,1cyclictest14338-21rm00:28:120
2332799230,0cyclictest16997-21diskmemload21:38:270
2332799230,0cyclictest0-21swapper/022:05:540
23330992222,0cyclictest16997-21diskmemload21:25:033
23330992222,0cyclictest0-21swapper/323:10:103
23330992220,1cyclictest0-21swapper/321:16:143
2333099220,21cyclictest27820-21kworker/u8:021:55:523
2333099220,21cyclictest16997-21diskmemload00:30:083
2333099220,21cyclictest10548-21apt-get00:25:193
2333099220,21cyclictest0-21swapper/322:20:333
2333099220,21cyclictest0-21swapper/322:06:053
2333099220,21cyclictest0-21swapper/321:20:203
2333099220,21cyclictest0-21swapper/320:20:213
2333099220,1cyclictest9077-21apt-get22:00:193
2333099220,1cyclictest2284-21ssh00:17:033
2333099220,1cyclictest12448-21apt-get21:00:193
2333099220,1cyclictest0-21swapper/323:27:383
2333099220,1cyclictest0-21swapper/300:21:283
2333099220,0cyclictest0-21swapper/322:14:133
2333099220,0cyclictest0-21swapper/321:38:033
23329992222,0cyclictest0-21swapper/221:45:422
23329992221,0cyclictest16997-21diskmemload23:45:062
23329992220,1cyclictest607-21in:imuxsock19:55:002
23329992220,1cyclictest20132-21cron20:10:012
23329992220,1cyclictest0-21swapper/222:32:052
23329992220,1cyclictest0-21swapper/221:30:252
23329992220,1cyclictest0-21swapper/220:15:182
2332999220,21cyclictest0-21swapper/223:20:132
2332999220,21cyclictest0-21swapper/222:55:142
2332999220,21cyclictest0-21swapper/222:45:192
2332999220,21cyclictest0-21swapper/221:41:172
2332999220,21cyclictest0-21swapper/220:35:202
2332999220,21cyclictest0-21swapper/219:30:022
2332999220,1cyclictest21754-21ssh00:35:162
2332999220,1cyclictest0-21swapper/223:51:392
2332999220,1cyclictest0-21swapper/223:11:252
23328992222,0cyclictest0-21swapper/123:40:281
23328992222,0cyclictest0-21swapper/122:27:111
23328992222,0cyclictest0-21swapper/100:11:191
23328992219,2cyclictest31871-21sh22:20:191
23328992219,2cyclictest12133-21ssh22:02:091
2332899220,21cyclictest27654-21apt-get20:25:201
2332899220,21cyclictest25523-21ssh23:11:101
2332899220,21cyclictest16989-21cron21:10:011
2332899220,21cyclictest0-21swapper/122:05:121
2332899220,21cyclictest0-21swapper/121:00:021
2332899220,1cyclictest5573-21ssh22:54:361
2332899220,1cyclictest0-21swapper/121:25:041
2332899220,1cyclictest0-21swapper/100:35:191
23327992222,0cyclictest13182-21apt-get19:55:190
23327992222,0cyclictest0-21swapper/021:48:480
23327992221,1cyclictest30187-21ssh21:50:160
23327992221,1cyclictest23980-21kernelversion23:10:240
23327992220,2cyclictest26483-21sh21:16:570
23327992220,1cyclictest26718-21diskstats22:15:210
23327992220,1cyclictest24059-21ssh00:36:060
2332799220,22cyclictest0-21swapper/023:01:130
2332799220,21cyclictest3303-21latency_hist19:35:000
2332799220,21cyclictest27427-21sh22:45:050
2332799220,21cyclictest19427-21cstates21:40:200
2332799220,21cyclictest0-21swapper/000:05:230
2332799220,1cyclictest29785-21munin-run20:30:000
2332799220,1cyclictest27629-21ssh00:10:300
2332799220,1cyclictest24706-21ssh22:41:330
2332799220,1cyclictest23293-21diskmemload23:39:150
2332799220,0cyclictest13755-21awk23:30:210
23330992120,1cyclictest5500-21sh23:51:093
23330992120,1cyclictest16997-21diskmemload23:05:163
2333099210,21cyclictest0-21swapper/323:40:133
2333099210,1cyclictest11751-21perf21:00:013
2332999210,20cyclictest5997-21apt-get19:40:182
2332999210,1cyclictest0-21swapper/200:05:592
2332999210,0cyclictest9641-21apt-get20:55:202
23328992121,0cyclictest0-21swapper/121:55:201
23328992121,0cyclictest0-21swapper/120:15:181
23328992120,0cyclictest0-21swapper/123:25:201
23328992119,1cyclictest28410-21apt-get19:20:191
2332799210,21cyclictest0-21swapper/020:45:200
2332799210,21cyclictest0-21swapper/000:20:150
2332799210,20cyclictest0-21swapper/019:10:200
23330992020,0cyclictest23592-21apt-get19:10:193
23330992020,0cyclictest0-21swapper/321:05:183
23330992020,0cyclictest0-21swapper/320:50:183
23330992020,0cyclictest0-21swapper/319:50:183
23330992020,0cyclictest0-21swapper/319:40:183
2333099200,20cyclictest0-21swapper/320:05:193
23329992020,0cyclictest0-21swapper/220:25:202
23329992019,1cyclictest25260-21apt-get20:20:212
23329992018,1cyclictest0-21swapper/220:50:002
2332999200,0cyclictest0-21swapper/219:45:182
2332999200,0cyclictest0-21swapper/219:25:182
23328992020,0cyclictest0-21swapper/120:05:191
23328992020,0cyclictest0-21swapper/119:30:161
23327992020,0cyclictest0-21swapper/021:00:190
23327992020,0cyclictest0-21swapper/020:20:210
23327992020,0cyclictest0-21swapper/019:40:170
23327992018,1cyclictest10498-21awk19:50:000
2332799200,0cyclictest0-21swapper/020:55:200
23329991919,0cyclictest20439-21apt-get20:10:202
23329991919,0cyclictest0-21swapper/219:15:192
23329991917,1cyclictest30536-21awk19:25:012
23329991917,1cyclictest14625-21perf21:05:012
23328991919,0cyclictest0-21swapper/120:55:201
23328991919,0cyclictest0-21swapper/120:30:191
23328991919,0cyclictest0-21swapper/120:00:191
23327991919,0cyclictest0-21swapper/019:20:190
2333099182,15cyclictest5671-21cron19:40:003
23330991817,1cyclictest15652-21apt-get20:00:193
23330991816,1cyclictest22830-21apt-get20:15:183
23328991815,2cyclictest2141-21cron20:40:001
23327991818,0cyclictest0-21swapper/020:15:170
23330991717,0cyclictest0-21swapper/320:45:203
23330991717,0cyclictest0-21swapper/319:55:193
23329991717,0cyclictest2452-21apt-get20:40:172
23329991717,0cyclictest0-21swapper/219:55:192
23328991717,0cyclictest8392-21apt-get19:45:181
23328991717,0cyclictest0-21swapper/120:45:201
23328991716,0cyclictest7241-21apt-get20:50:181
23328991715,1cyclictest0-21swapper/119:10:231
23327991717,0cyclictest0-21swapper/020:50:180
23330991616,0cyclictest0-21swapper/320:40:173
23330991616,0cyclictest0-21swapper/319:30:023
23330991616,0cyclictest0-21swapper/319:15:183
23330991614,1cyclictest32183-21perf20:35:003
2333099160,15cyclictest0-21swapper/319:45:183
2332999160,15cyclictest24394-21irqstats19:10:232
23328991616,0cyclictest0-21swapper/120:10:201
23328991614,1cyclictest0-21swapper/119:35:201
23327991614,1cyclictest0-21swapper/020:10:200
23330991513,1cyclictest0-21swapper/320:25:233
23329991513,1cyclictest0-21swapper/219:35:202
23328991515,0cyclictest0-21swapper/119:50:191
23327991515,0cyclictest0-21swapper/019:35:190
23327991514,1cyclictest17767-21perf20:05:010
2332799150,13cyclictest0-21swapper/020:40:160
23330991414,0cyclictest0-21swapper/320:10:203
2333099140,14cyclictest0-21swapper/320:35:503
2333099140,0cyclictest0-21swapper/319:20:033
23329991414,0cyclictest0-21swapper/220:50:182
2332999140,14cyclictest0-21swapper/221:05:062
2332999140,14cyclictest0-21swapper/220:30:452
23328991414,0cyclictest0-21swapper/120:40:031
2332899140,14cyclictest0-21swapper/120:20:031
2332899140,0cyclictest0-21swapper/119:40:201
23327991414,0cyclictest0-21swapper/021:05:010
23327991414,0cyclictest0-21swapper/019:25:120
23327991412,1cyclictest10785-21apt-get19:50:180
2332799140,0cyclictest0-21swapper/019:15:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional