You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 06:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri May 01, 2026 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2317899276,21cyclictest30370-21kworker/1:000:30:381
23179992626,0cyclictest271rcuc/221:28:432
2317999261,2cyclictest28-21ksoftirqd/221:10:212
2317999250,24cyclictest0-21swapper/200:33:172
23177992524,1cyclictest0-21swapper/000:00:290
2317799250,24cyclictest16805-21diskmemload21:14:560
2317799250,23cyclictest27783-21ssh00:30:500
2317799250,23cyclictest16805-21diskmemload21:42:120
2317799250,1cyclictest13275-21ssh00:16:050
2318199240,24cyclictest0-21swapper/300:28:073
2318199240,23cyclictest28854-21sh21:51:163
2318199240,23cyclictest16805-21diskmemload23:02:013
2318199240,23cyclictest0-21swapper/321:48:453
2318199240,1cyclictest16805-21diskmemload22:27:533
2317999240,23cyclictest0-21swapper/223:49:192
2317999240,23cyclictest0-21swapper/223:27:562
2317999240,23cyclictest0-21swapper/223:06:022
2317999240,23cyclictest0-21swapper/221:32:522
2317999240,0cyclictest0-21swapper/222:35:322
2317999240,0cyclictest0-21swapper/221:38:372
23178992424,0cyclictest6583-21kworker/1:222:53:561
23178992424,0cyclictest0-21swapper/121:28:411
2317899240,23cyclictest31036-21sh23:28:441
2317899240,23cyclictest21628-21awk23:20:011
2317899240,23cyclictest0-21swapper/121:55:351
23177992424,0cyclictest0-21swapper/022:59:260
23177992423,1cyclictest11605-21apt-get21:35:580
23177992423,1cyclictest0-21swapper/000:25:480
23177992422,1cyclictest0-21swapper/023:30:210
2317799240,2cyclictest24562-21/usr/sbin/munin22:51:050
2317799240,23cyclictest14773-21cron23:45:000
2317799240,1cyclictest0-21swapper/021:52:260
2317799240,0cyclictest0-21swapper/000:20:130
23181992323,0cyclictest16805-21diskmemload22:50:373
23181992323,0cyclictest16805-21diskmemload00:31:493
23181992323,0cyclictest0-21swapper/300:21:013
23181992322,1cyclictest0-21swapper/323:27:513
23181992320,2cyclictest22406-21ssh21:15:213
2318199230,23cyclictest16805-21diskmemload23:56:073
2318199230,23cyclictest0-21swapper/323:40:443
2318199230,23cyclictest0-21swapper/322:07:483
2318199230,23cyclictest0-21swapper/321:33:003
2318199230,23cyclictest0-21swapper/321:11:053
2318199230,23cyclictest0-21swapper/300:14:153
2318199230,22cyclictest7177-21ssh22:02:003
2318199230,22cyclictest27653-21sh22:22:093
2318199230,22cyclictest22395-21ssh22:16:533
2318199230,22cyclictest1679-21ntpq21:26:043
2318199230,22cyclictest1553-21sh21:56:153
2318199230,22cyclictest14224-21memory22:41:043
2318199230,22cyclictest10511-21ssh23:09:173
2318199230,22cyclictest0-21swapper/322:56:473
2318199230,0cyclictest16805-21diskmemload22:31:483
2318199230,0cyclictest16805-21diskmemload00:19:523
23179992323,0cyclictest16805-21diskmemload22:28:182
23179992323,0cyclictest0-21swapper/221:47:592
23179992323,0cyclictest0-21swapper/221:44:232
23179992321,1cyclictest0-21swapper/200:05:562
23179992320,2cyclictest25187-21sh22:20:562
23179992319,0cyclictest0-21swapper/223:13:022
2317999231,1cyclictest271rcuc/222:00:012
2317999230,23cyclictest0-21swapper/223:59:032
2317999230,22cyclictest22532-21sh21:15:312
2317999230,22cyclictest2212-21sh21:57:192
2317999230,22cyclictest16805-21diskmemload22:52:152
2317999230,22cyclictest16805-21diskmemload22:42:372
2317999230,22cyclictest16805-21diskmemload00:25:292
2317999230,22cyclictest0-21swapper/222:47:212
2317999230,22cyclictest0-21swapper/221:21:042
2317999230,0cyclictest1741-21apt-get23:01:012
2317999230,0cyclictest0-21swapper/223:17:342
2317999230,0cyclictest0-21swapper/222:55:542
2317999230,0cyclictest0-21swapper/221:51:042
2317999230,0cyclictest0-21swapper/200:36:032
2317999230,0cyclictest0-21swapper/200:12:232
23178992323,0cyclictest16805-21diskmemload23:56:111
23178992323,0cyclictest0-21swapper/122:32:351
23178992323,0cyclictest0-21swapper/122:15:031
23178992323,0cyclictest0-21swapper/121:51:501
23178992323,0cyclictest0-21swapper/121:34:251
23178992323,0cyclictest0-21swapper/100:00:131
23178992321,1cyclictest31815-21ssh22:57:491
23178992321,1cyclictest29555-21sh20:30:001
23178992321,1cyclictest26912-21munin-run23:25:011
23178992321,1cyclictest16805-21diskmemload23:11:031
23178992321,1cyclictest0-21swapper/122:39:511
23178992321,1cyclictest0-21swapper/121:10:081
23178992321,1cyclictest0-21swapper/100:25:511
2317899230,2cyclictest4803-21rm23:03:171
2317899230,23cyclictest0-21swapper/123:49:521
2317899230,22cyclictest9408-21sh22:05:121
2317899230,22cyclictest2444-21ssh23:31:451
2317899230,22cyclictest22878-21sh21:15:591
2317899230,22cyclictest18238-21ssh21:41:061
2317899230,22cyclictest1576-21ssh00:05:271
2317899230,22cyclictest10153-21sh00:13:161
2317899230,1cyclictest17316-21apt-get00:21:031
2317899230,0cyclictest0-21swapper/122:46:141
2317899230,0cyclictest0-21swapper/122:21:591
23177992323,0cyclictest16805-21diskmemload21:27:590
23177992323,0cyclictest0-21swapper/021:20:580
23177992322,1cyclictest0-21swapper/022:47:250
23177992321,1cyclictest9933-21idleruntime-cro19:50:000
23177992321,1cyclictest774-21runrttasks21:15:360
23177992321,1cyclictest0-21swapper/023:24:430
23177992321,1cyclictest0-21swapper/023:05:010
23177992320,2cyclictest6885-21ssh22:01:320
23177992320,2cyclictest2706-21apt19:35:000
2317799230,2cyclictest17408-21ssh23:15:580
2317799230,2cyclictest12237-21rm22:06:530
2317799230,23cyclictest0-21swapper/023:53:460
2317799230,23cyclictest0-21swapper/023:28:280
2317799230,23cyclictest0-21swapper/022:37:250
2317799230,22cyclictest9196-21chrt00:11:430
2317799230,22cyclictest8498-21ssh21:32:040
2317799230,22cyclictest62550irq/127-eno123:48:200
2317799230,22cyclictest4524-21rm00:39:350
2317799230,22cyclictest2480-21ssh00:05:580
2317799230,22cyclictest0-21swapper/022:30:130
2317799230,1cyclictest16805-21diskmemload21:46:380
2317799230,0cyclictest16805-21diskmemload22:43:490
23181992222,0cyclictest16805-21diskmemload23:35:553
23181992222,0cyclictest16805-21diskmemload21:22:483
23181992222,0cyclictest0-21swapper/323:16:413
23181992221,0cyclictest3040-21apt-get19:35:583
23181992220,1cyclictest20111-21sh23:50:073
23181992220,1cyclictest0-21swapper/322:50:003
23181992220,1cyclictest0-21swapper/322:35:593
2318199220,22cyclictest0-21swapper/300:36:453
2318199220,21cyclictest16805-21diskmemload23:20:333
2318199220,21cyclictest0-21swapper/323:12:253
2318199220,21cyclictest0-21swapper/321:40:333
2318199220,21cyclictest0-21swapper/300:03:303
2318199220,1cyclictest16805-21diskmemload23:45:503
2318199220,0cyclictest16805-21diskmemload00:07:313
23179992222,0cyclictest16805-21diskmemload23:30:262
23179992222,0cyclictest0-21swapper/223:50:082
23179992222,0cyclictest0-21swapper/223:41:242
23179992222,0cyclictest0-21swapper/223:35:272
23179992222,0cyclictest0-21swapper/222:14:342
23179992222,0cyclictest0-21swapper/200:20:342
23179992220,1cyclictest4374-21idleruntime-cro20:45:012
23179992220,1cyclictest0-21swapper/222:18:542
23179992220,1cyclictest0-21swapper/222:05:082
23179992220,1cyclictest0-21swapper/220:35:572
23179992219,2cyclictest7523-21awk19:45:002
2317999220,21cyclictest21898-21ssh23:20:082
2317999220,21cyclictest16805-21diskmemload00:16:182
2317999220,21cyclictest0-21swapper/221:05:592
2317999220,21cyclictest0-21swapper/219:55:012
2317999220,21cyclictest0-21swapper/219:45:582
2317999220,21cyclictest0-21swapper/219:10:582
2317999220,1cyclictest30322-21latency_hist19:25:002
23178992222,0cyclictest16805-21diskmemload23:05:101
23178992222,0cyclictest0-21swapper/123:40:061
23178992221,1cyclictest30875-21perl22:26:021
23178992220,1cyclictest0-21swapper/123:50:241
23178992220,1cyclictest0-21swapper/100:16:061
2317899220,22cyclictest0-21swapper/121:37:361
2317899220,21cyclictest0-21swapper/122:10:041
2317899220,21cyclictest0-21swapper/122:00:581
2317899220,21cyclictest0-21swapper/100:36:001
2317899220,1cyclictest2697-21cron19:35:001
2317899220,1cyclictest24858-21ssh21:48:001
2317899220,1cyclictest0-21swapper/123:37:091
2317899220,1cyclictest0-21swapper/121:21:081
2317899220,0cyclictest0-21swapper/122:40:281
23177992221,1cyclictest28264-21apt-get19:20:580
23177992220,2cyclictest5829-21/usr/sbin/munin23:36:030
23177992220,1cyclictest6220-21ssh23:05:080
23177992220,1cyclictest32302-21apt-get20:35:570
2317799220,22cyclictest16805-21diskmemload23:14:220
2317799220,22cyclictest0-21swapper/022:25:190
2317799220,21cyclictest5459-21apt-get19:40:580
2317799220,21cyclictest31782-21sh21:55:360
2317799220,21cyclictest24907-21sh22:20:300
2317799220,21cyclictest16805-21diskmemload23:55:040
2317799220,1cyclictest0-21swapper/022:15:540
23181992119,1cyclictest30656-21apt-get19:25:583
2318199210,21cyclictest0-21swapper/323:30:473
2318199210,21cyclictest0-21swapper/322:10:373
2318199210,20cyclictest25119-21apt-get20:20:583
23179992120,0cyclictest0-21swapper/220:00:572
2317999210,1cyclictest0-21swapper/200:00:022
2317999210,0cyclictest0-21swapper/220:05:582
23178992120,1cyclictest0-21swapper/119:40:581
23178992119,1cyclictest14765-21wc20:00:011
23178992119,1cyclictest14765-21wc20:00:011
2317899210,20cyclictest0-21swapper/120:30:561
2317899210,0cyclictest0-21swapper/120:00:571
2317899210,0cyclictest0-21swapper/119:50:581
23177992120,1cyclictest16805-21diskmemload22:10:050
23177992119,1cyclictest25798-21sshd19:15:480
2317799210,1cyclictest622-21gdbus20:45:010
2317799210,1cyclictest23451-21apt-get19:10:580
2317799210,1cyclictest16806-21cron21:10:000
23181992020,0cyclictest0-21swapper/319:30:573
2318199200,0cyclictest0-21swapper/321:00:023
23179992020,0cyclictest0-21swapper/221:01:002
23179992020,0cyclictest0-21swapper/219:55:592
23179992020,0cyclictest0-21swapper/219:55:592
23179992019,0cyclictest0-21swapper/220:15:582
23179992018,1cyclictest0-21swapper/220:25:002
2317999200,0cyclictest0-21swapper/220:50:592
2317999200,0cyclictest0-21swapper/220:45:592
2317999200,0cyclictest0-21swapper/220:45:592
23178992020,0cyclictest0-21swapper/120:20:581
23178992013,4cyclictest0-21swapper/120:50:591
2317899200,0cyclictest0-21swapper/121:05:581
23177992019,1cyclictest27514-21apt-get20:26:000
23177992019,1cyclictest12697-21apt-get19:55:580
23177992019,1cyclictest12697-21apt-get19:55:580
23181991919,0cyclictest7526-21perf19:45:003
23181991919,0cyclictest0-21swapper/320:15:583
23181991919,0cyclictest0-21swapper/320:05:583
23181991917,1cyclictest17191-21awk20:05:013
23181991916,2cyclictest9177-21awk20:55:013
23179991917,1cyclictest606-21dbus-daemon19:20:012
2317999190,0cyclictest0-21swapper/219:36:162
23178991919,0cyclictest0-21swapper/121:01:001
23178991919,0cyclictest0-21swapper/120:56:001
23178991919,0cyclictest0-21swapper/119:45:581
23178991919,0cyclictest0-21swapper/119:35:581
23178991916,2cyclictest25523-21latency_hist19:15:011
23177991919,0cyclictest11557-21apt-get21:00:020
23177991919,0cyclictest0-21swapper/020:45:590
23177991919,0cyclictest0-21swapper/020:45:590
23177991917,1cyclictest17903-21apt-get20:05:580
23179991818,0cyclictest0-21swapper/220:10:582
23178991818,0cyclictest0-21swapper/120:35:571
23178991818,0cyclictest0-21swapper/119:15:581
23177991818,0cyclictest0-21swapper/020:10:580
23177991816,1cyclictest5107-21turbostat19:40:000
23177991816,1cyclictest12372-21latency_hist19:55:010
23181991717,0cyclictest0-21swapper/321:05:593
23181991717,0cyclictest0-21swapper/320:30:563
23181991715,1cyclictest0-21swapper/319:50:003
2317999170,17cyclictest0-21swapper/219:28:272
23178991717,0cyclictest0-21swapper/120:40:581
23178991717,0cyclictest0-21swapper/120:15:581
23178991717,0cyclictest0-21swapper/120:05:021
23177991717,0cyclictest9500-21apt-get20:56:000
23177991717,0cyclictest0-21swapper/020:15:580
23177991715,1cyclictest9179-21munin-run20:55:010
23181991615,1cyclictest20297-21apt-get20:10:583
23181991615,0cyclictest0-21swapper/319:20:583
23181991614,1cyclictest4391-21munin-run20:45:013
23181991614,1cyclictest1985-21munin-run20:40:003
2318199160,15cyclictest11152-21memory19:51:033
23179991614,1cyclictest28416-21memory20:26:042
2317899160,2cyclictest5436-21ethtool20:46:021
2317899160,2cyclictest5436-21ethtool20:46:021
2317899160,15cyclictest0-21swapper/120:10:431
23181991515,0cyclictest0-21swapper/320:45:593
23181991515,0cyclictest0-21swapper/320:45:593
23181991513,1cyclictest27941-21sendmail-msp19:20:013
23181991513,1cyclictest0-21swapper/321:00:013
2318199150,14cyclictest24777-21plymouth19:11:053
2318199150,14cyclictest0-21swapper/319:56:023
2318199150,14cyclictest0-21swapper/319:56:023
23178991513,1cyclictest0-21swapper/119:25:581
2318199140,14cyclictest0-21swapper/320:25:173
2317999140,14cyclictest0-21swapper/220:30:092
2317999140,14cyclictest0-21swapper/219:30:542
2317899140,14cyclictest0-21swapper/119:20:411
23177991414,0cyclictest0-21swapper/020:20:110
2317799140,14cyclictest0-21swapper/020:00:040
2317799140,14cyclictest0-21swapper/019:25:050
2317799140,0cyclictest0-21swapper/020:31:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional