You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 14:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat May 02, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2647999280,1cyclictest12967-21munin-run21:35:012
2647999270,26cyclictest0-21swapper/223:02:372
2648399260,25cyclictest313-21systemd-journal00:10:013
2648399260,1cyclictest31124-21sh00:30:023
2647999260,25cyclictest0-21swapper/223:35:472
2647999260,25cyclictest0-21swapper/221:46:032
2647999260,25cyclictest0-21swapper/220:01:032
2647999260,25cyclictest0-21swapper/200:36:562
2647999260,25cyclictest0-21swapper/200:21:162
2647999260,1cyclictest23740-21ssh21:13:142
26477992624,1cyclictest25889-21expr22:51:060
2648399250,24cyclictest20900-21expr21:11:043
26479992521,3cyclictest19751-21diskmemload21:38:152
26479992520,4cyclictest19751-21diskmemload22:58:012
2647999250,24cyclictest27320-21ssh23:55:112
2647999250,24cyclictest0-21swapper/200:19:142
2647999250,21cyclictest0-21swapper/223:12:002
2647999250,1cyclictest4904-21ssh23:32:072
2647999250,1cyclictest0-21swapper/222:52:322
2647999250,1cyclictest0-21swapper/221:51:382
2647999250,0cyclictest0-21swapper/221:20:492
26478992525,0cyclictest0-21swapper/122:49:571
26483992423,1cyclictest19751-21diskmemload23:00:093
26483992422,1cyclictest19751-21diskmemload00:02:513
2648399240,23cyclictest2133-21ssh22:26:213
2648399240,23cyclictest21126-21ssh00:20:173
2648399240,1cyclictest0-21swapper/323:56:263
26479992421,2cyclictest622-21gdbus22:40:002
26479992419,4cyclictest26551-21ssh00:25:482
26479992419,4cyclictest12139-21ssh23:08:572
26479992419,4cyclictest0-21swapper/200:06:112
26479992418,1cyclictest0-21swapper/222:46:092
2647999240,24cyclictest0-21swapper/222:34:212
2647999240,23cyclictest19751-21diskmemload00:02:392
2647999240,23cyclictest0-21swapper/222:11:262
2647999240,23cyclictest0-21swapper/221:19:182
2647999240,0cyclictest0-21swapper/223:17:562
2647999240,0cyclictest0-21swapper/222:24:182
2647999240,0cyclictest0-21swapper/200:31:542
2647899240,23cyclictest9739-21cron20:50:011
2647899240,23cyclictest19751-21diskmemload21:34:171
2647899240,23cyclictest0-21swapper/123:12:061
2647899240,0cyclictest0-21swapper/122:13:031
26477992423,1cyclictest0-21swapper/022:01:060
26477992422,1cyclictest62550irq/127-eno123:21:420
26477992420,3cyclictest24717-21rm21:14:510
2647799240,24cyclictest0-21swapper/021:57:080
2647799240,24cyclictest0-21swapper/000:15:150
2647799240,22cyclictest18345-21rm22:42:530
2647799240,0cyclictest0-21swapper/021:53:350
26483992323,0cyclictest19751-21diskmemload23:44:413
26483992323,0cyclictest0-21swapper/323:32:443
26483992323,0cyclictest0-21swapper/322:55:323
26483992323,0cyclictest0-21swapper/322:16:363
26483992323,0cyclictest0-21swapper/322:03:283
26483992321,1cyclictest19751-21diskmemload22:41:083
26483992321,1cyclictest19751-21diskmemload00:10:013
26483992321,1cyclictest19718-21ssh21:41:063
26483992321,1cyclictest10079-21ssh22:35:373
26483992321,1cyclictest0-21swapper/322:46:303
26483992321,1cyclictest0-21swapper/300:26:033
26483992320,2cyclictest2564-21sh20:35:003
2648399230,2cyclictest28595-21rm23:24:373
2648399230,2cyclictest25282-21ssh22:50:273
2648399230,23cyclictest0-21swapper/321:17:203
2648399230,22cyclictest4383-21ssh00:35:553
2648399230,22cyclictest30133-21sh22:22:343
2648399230,22cyclictest22343-21ssh23:17:493
2648399230,22cyclictest19751-21diskmemload23:35:313
2648399230,1cyclictest151rcu_preempt21:46:313
2648399230,0cyclictest0-21swapper/321:22:543
26479992323,0cyclictest0-21swapper/222:15:322
26479992320,3cyclictest0-21swapper/222:44:122
26479992320,2cyclictest20234-21ssh23:47:112
26479992318,1cyclictest0-21swapper/223:50:402
26479992317,1cyclictest8907-21ssh22:01:162
2647999230,23cyclictest0-21swapper/221:27:482
2647999230,22cyclictest3920-21ssh21:56:332
2647999230,0cyclictest0-21swapper/223:41:262
2647999230,0cyclictest0-21swapper/223:27:232
2647999230,0cyclictest0-21swapper/221:42:012
26478992323,0cyclictest19751-21diskmemload22:52:251
26478992323,0cyclictest19751-21diskmemload00:19:261
26478992323,0cyclictest19751-21diskmemload00:11:521
26478992323,0cyclictest0-21swapper/123:41:241
26478992323,0cyclictest0-21swapper/122:00:111
26478992323,0cyclictest0-21swapper/100:35:531
26478992321,1cyclictest0-21swapper/123:26:101
26478992321,1cyclictest0-21swapper/100:00:461
26478992320,2cyclictest25649-21ssh23:21:091
2647899230,23cyclictest19751-21diskmemload21:10:511
2647899230,23cyclictest0-21swapper/123:36:331
2647899230,23cyclictest0-21swapper/123:15:101
2647899230,23cyclictest0-21swapper/122:37:081
2647899230,22cyclictest19751-21diskmemload00:26:181
2647899230,22cyclictest0-21swapper/121:26:211
2647899230,22cyclictest0-21swapper/121:21:351
2647899230,22cyclictest0-21swapper/119:51:031
2647899230,0cyclictest19751-21diskmemload23:30:461
2647899230,0cyclictest0-21swapper/123:07:341
2647899230,0cyclictest0-21swapper/122:41:361
2647899230,0cyclictest0-21swapper/121:36:581
26477992323,0cyclictest0-21swapper/023:35:300
26477992323,0cyclictest0-21swapper/021:43:090
26477992323,0cyclictest0-21swapper/021:32:580
26477992323,0cyclictest0-21swapper/021:27:380
26477992322,1cyclictest62550irq/127-eno100:06:470
26477992322,1cyclictest1998-21ssh23:30:170
26477992322,1cyclictest19751-21diskmemload23:26:180
26477992322,1cyclictest14412-21rm22:39:540
26477992321,1cyclictest62550irq/127-eno100:01:250
26477992321,1cyclictest31691-21munin-plugin-st22:25:010
26477992321,1cyclictest0-21swapper/000:21:510
26477992320,2cyclictest31872-21cut22:56:100
26477992320,2cyclictest19751-21diskmemload00:31:580
26477992320,2cyclictest0-21swapper/023:46:260
26477992319,3cyclictest18903-21apt-get23:15:020
2647799230,2cyclictest8508-21latency_hist19:40:000
2647799230,2cyclictest20234-21ssh22:45:370
2647799230,23cyclictest0-21swapper/023:10:130
2647799230,22cyclictest31909-21sh22:25:020
2647799230,22cyclictest19751-21diskmemload21:38:250
2647799230,21cyclictest25377-21cut21:46:100
2647799230,0cyclictest62550irq/127-eno122:15:580
2647799230,0cyclictest29199-21ssh00:27:180
2647799230,0cyclictest19751-21diskmemload00:38:550
2647799230,0cyclictest0-21swapper/023:58:580
2647799230,0cyclictest0-21swapper/021:23:450
2647799230,0cyclictest0-21swapper/021:18:120
26483992222,0cyclictest0-21swapper/323:54:043
26483992222,0cyclictest0-21swapper/323:10:033
26483992222,0cyclictest0-21swapper/322:05:233
26483992222,0cyclictest0-21swapper/321:57:323
26483992222,0cyclictest0-21swapper/321:28:243
26483992221,1cyclictest8430-21ssh21:30:383
26483992221,1cyclictest0-21swapper/300:15:143
26483992220,1cyclictest29765-21apt-get23:26:043
26483992220,1cyclictest17527-21sh23:45:463
26483992220,1cyclictest0-21swapper/321:50:233
2648399220,21cyclictest4986-21sendmail-msp20:40:003
2648399220,21cyclictest25319-21perf20:15:013
2648399220,21cyclictest19751-21diskmemload21:35:303
2648399220,21cyclictest10105-21apt-get20:51:043
2648399220,1cyclictest10617-21sh23:06:333
2648399220,0cyclictest0-21swapper/322:11:253
26479992221,1cyclictest0-21swapper/219:41:052
26479992220,1cyclictest13275-21ssh22:06:112
26479992220,1cyclictest10903-21sh00:10:212
2647999220,21cyclictest23245-21apt-get20:11:032
2647999220,1cyclictest28021-21ssh23:23:462
26478992222,0cyclictest0-21swapper/121:50:131
26478992221,1cyclictest0-21swapper/122:17:081
26478992221,0cyclictest16984-21apt-get21:05:031
26478992220,2cyclictest30172-21sh22:22:371
26478992220,2cyclictest26154-21ssh23:53:381
26478992220,1cyclictest25389-21ssh21:46:101
26478992220,1cyclictest18336-21ssh21:40:091
26478992220,1cyclictest0-21swapper/123:46:031
26478992220,1cyclictest0-21swapper/100:06:281
2647899220,21cyclictest3184-21ssh23:00:371
2647899220,21cyclictest0-21swapper/122:55:041
2647899220,21cyclictest0-21swapper/100:31:541
2647899220,1cyclictest27950-21ssh21:16:491
2647899220,1cyclictest10926-21munin-run19:45:001
2647899220,1cyclictest0-21swapper/122:30:171
2647899220,1cyclictest0-21swapper/119:40:001
2647899220,0cyclictest0-21swapper/123:55:161
26477992220,1cyclictest0-21swapper/022:30:360
2647799220,22cyclictest19751-21diskmemload22:13:080
2647799220,21cyclictest11623-21ssh22:05:170
2647799220,21cyclictest0-21swapper/023:40:230
2647799220,21cyclictest0-21swapper/000:10:020
2647799220,1cyclictest7386-21ssh23:04:350
2647799220,0cyclictest0-21swapper/023:05:140
2648399210,1cyclictest18120-21munin-run20:00:013
2648399210,0cyclictest11249-21apt-get19:46:043
2648399210,0cyclictest0-21swapper/322:31:043
2648399210,0cyclictest0-21swapper/320:01:043
2648399210,0cyclictest0-21swapper/319:55:013
26479992120,0cyclictest5313-21apt-get20:41:042
2647999210,21cyclictest0-21swapper/220:51:042
2647999210,0cyclictest0-21swapper/220:36:052
26478992121,0cyclictest30515-21apt-get20:26:021
2647899210,0cyclictest0-21swapper/122:06:051
2647899210,0cyclictest0-21swapper/100:20:021
26477992121,0cyclictest23245-21apt-get20:11:030
26477992119,1cyclictest0-21swapper/020:45:010
2647799210,0cyclictest12500-21apt-get20:56:040
26483992020,0cyclictest0-21swapper/319:31:023
26479992020,0cyclictest0-21swapper/221:01:032
26479992020,0cyclictest0-21swapper/220:31:022
26479992020,0cyclictest0-21swapper/219:31:022
26479992019,1cyclictest29183-21apt-get19:16:002
26479992018,1cyclictest9766-21ls20:50:012
26479992018,1cyclictest1634-21apt-get19:26:022
2647999200,0cyclictest0-21swapper/220:26:022
26478992020,0cyclictest0-21swapper/120:51:031
26478992020,0cyclictest0-21swapper/120:31:021
26478992020,0cyclictest0-21swapper/120:06:031
26478992020,0cyclictest0-21swapper/119:56:041
26478992019,1cyclictest0-21swapper/120:11:031
2647899200,0cyclictest0-21swapper/120:41:041
26477992019,1cyclictest0-21swapper/019:56:040
2647799200,20cyclictest0-21swapper/020:36:040
2647799200,19cyclictest0-21swapper/020:46:010
26483991919,0cyclictest0-21swapper/320:16:033
26483991919,0cyclictest0-21swapper/319:41:043
26483991918,1cyclictest6436-21apt-get19:36:023
26483991918,0cyclictest0-21swapper/321:01:033
26483991917,1cyclictest313-21systemd-journal19:30:003
26483991917,1cyclictest1289-21awk19:25:013
26479991919,0cyclictest0-21swapper/219:51:032
26479991919,0cyclictest0-21swapper/219:36:022
2647999190,1cyclictest606-21dbus-daemon19:50:002
26478991917,1cyclictest6118-21dump-pmu-power19:35:001
26477991919,0cyclictest0-21swapper/020:01:030
26477991918,1cyclictest0-21swapper/021:01:030
26483991818,0cyclictest0-21swapper/319:11:023
2648399180,0cyclictest0-21swapper/320:44:533
26479991818,0cyclictest0-21swapper/219:56:042
26477991818,0cyclictest0-21swapper/019:21:020
26477991817,0cyclictest25639-21apt-get20:16:030
26477991817,0cyclictest0-21swapper/020:21:030
2647799180,0cyclictest0-21swapper/019:31:020
26483991717,0cyclictest30515-21apt-get20:26:033
26483991716,0cyclictest0-21swapper/320:06:033
2647999170,14cyclictest313-21systemd-journal20:19:562
26478991717,0cyclictest0-21swapper/119:26:021
26478991715,1cyclictest16985-21dump-pmu-power21:05:011
26477991717,0cyclictest26792-21apt-get19:11:020
26477991717,0cyclictest0-21swapper/020:06:030
26477991717,0cyclictest0-21swapper/019:51:030
26483991616,0cyclictest28068-21apt-get20:21:033
2648399160,16cyclictest0-21swapper/319:16:003
26479991616,0cyclictest0-21swapper/221:06:022
26479991616,0cyclictest0-21swapper/220:21:032
26478991616,0cyclictest0-21swapper/120:56:031
26478991616,0cyclictest0-21swapper/119:11:021
26478991614,1cyclictest0-21swapper/119:25:011
26478991613,2cyclictest13322-21munin-run19:50:001
26477991616,0cyclictest0-21swapper/020:26:020
26483991513,1cyclictest9782-21munin-run20:50:013
2648399150,1cyclictest19778-21pluginstate21:10:003
26479991513,1cyclictest21651-21irqstats20:06:062
2647999150,14cyclictest0-21swapper/220:56:112
26478991515,0cyclictest18457-21apt-get20:01:041
26477991515,0cyclictest0-21swapper/019:46:040
26477991514,1cyclictest0-21swapper/019:45:000
2648399140,14cyclictest0-21swapper/320:56:263
2647999140,14cyclictest0-21swapper/219:20:472
2647999140,0cyclictest0-21swapper/219:10:432
2647899140,14cyclictest0-21swapper/120:35:101
2647899140,14cyclictest0-21swapper/119:15:071
2647899140,0cyclictest0-21swapper/120:15:271
26477991414,0cyclictest0-21swapper/020:51:040
2647799140,14cyclictest0-21swapper/020:30:510
2647799140,0cyclictest0-21swapper/019:26:140
2647799140,0cyclictest0-21swapper/019:15:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional