You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 17:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Apr 30, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26526995114,1cyclictest0-21swapper/010:12:390
26531993735,1cyclictest20279-21lspci10:12:392
2652799253,21cyclictest22-21ksoftirqd/112:20:531
26526992523,1cyclictest0-21swapper/009:14:140
2652699250,24cyclictest0-21swapper/009:43:020
26533992422,1cyclictest17472-21sh12:15:583
2653399240,23cyclictest19378-21sh10:12:033
2653399240,1cyclictest18186-21diskmemload12:13:393
2653199241,0cyclictest0-21swapper/211:29:502
2653199240,23cyclictest2151-21if_eno112:00:592
26527992422,1cyclictest22-21ksoftirqd/109:28:181
2652799240,24cyclictest0-21swapper/110:13:211
2652799240,23cyclictest7555-21grep10:31:051
2652799240,23cyclictest18186-21diskmemload09:40:531
2652799240,1cyclictest18186-21diskmemload12:37:501
2652799240,0cyclictest18186-21diskmemload11:09:301
2652799240,0cyclictest18186-21diskmemload09:59:511
2652799240,0cyclictest0-21swapper/110:00:271
26526992421,2cyclictest13451-21apt07:50:000
2652699240,24cyclictest0-21swapper/012:21:340
2652699240,23cyclictest0-21swapper/009:29:410
2652699240,1cyclictest30493-21sh10:22:000
2652699240,0cyclictest18186-21diskmemload11:13:230
2652699240,0cyclictest18186-21diskmemload11:01:060
26533992323,0cyclictest3754-1kworker/u9:212:27:443
26533992323,0cyclictest0-21swapper/311:05:113
26533992321,1cyclictest313-21systemd-journal10:55:003
26533992321,1cyclictest15343-21ssh09:36:543
26533992321,1cyclictest0-21swapper/311:54:543
2653399230,23cyclictest0-21swapper/312:23:003
2653399230,23cyclictest0-21swapper/309:57:283
2653399230,22cyclictest8043-21ssh11:35:493
2653399230,22cyclictest4445-21ssh10:28:583
2653399230,22cyclictest24391-21ssh11:20:003
2653399230,22cyclictest18186-21diskmemload09:45:273
2653399230,22cyclictest10058-21runrttasks09:31:443
2653399230,1cyclictest19454-21kworker/u8:210:35:123
2653399230,0cyclictest0-21swapper/311:42:453
2653399230,0cyclictest0-21swapper/311:02:143
26531992323,0cyclictest0-21swapper/212:35:582
26531992323,0cyclictest0-21swapper/211:15:282
26531992323,0cyclictest0-21swapper/210:21:192
26531992323,0cyclictest0-21swapper/209:31:462
26531992320,2cyclictest28800-21sh09:50:582
26531992320,2cyclictest28303-21sh10:51:042
26531992320,2cyclictest19292-21awk11:15:012
26531992319,3cyclictest0-21swapper/212:20:172
2653199230,23cyclictest18186-21diskmemload10:37:542
2653199230,23cyclictest18186-21diskmemload10:34:012
2653199230,23cyclictest0-21swapper/212:31:172
2653199230,23cyclictest0-21swapper/211:35:532
2653199230,23cyclictest0-21swapper/210:45:242
2653199230,23cyclictest0-21swapper/209:47:122
2653199230,22cyclictest6327-21/usr/sbin/munin10:01:172
2653199230,22cyclictest23871-21ssh09:15:312
2653199230,22cyclictest17875-21ssh09:40:372
2653199230,0cyclictest18186-21diskmemload10:09:222
2653199230,0cyclictest0-21swapper/211:58:052
26527992323,0cyclictest18186-21diskmemload11:21:451
26527992323,0cyclictest0-21swapper/112:30:341
26527992323,0cyclictest0-21swapper/112:06:011
26527992323,0cyclictest0-21swapper/109:51:031
26527992322,1cyclictest22-21ksoftirqd/110:47:461
26527992322,1cyclictest15920-21ssh11:42:181
26527992322,1cyclictest0-21swapper/109:23:341
26527992321,1cyclictest0-21swapper/111:31:031
26527992321,1cyclictest0-21swapper/111:11:281
26527992321,1cyclictest0-21swapper/110:52:441
26527992321,1cyclictest0-21swapper/110:21:101
26527992321,1cyclictest0-21swapper/110:18:531
26527992321,1cyclictest0-21swapper/110:05:011
2652799231,21cyclictest10174-21apt-get10:35:021
2652799230,23cyclictest0-21swapper/112:27:101
2652799230,23cyclictest0-21swapper/111:55:421
2652799230,23cyclictest0-21swapper/111:18:411
2652799230,22cyclictest10195-21sh11:36:251
2652799230,0cyclictest0-21swapper/112:12:211
2652799230,0cyclictest0-21swapper/111:26:441
2652799230,0cyclictest0-21swapper/110:58:101
2652799230,0cyclictest0-21swapper/109:31:571
26526992323,0cyclictest62550irq/127-eno111:29:030
26526992323,0cyclictest18186-21diskmemload10:31:340
26526992323,0cyclictest0-21swapper/011:09:240
26526992323,0cyclictest0-21swapper/010:27:050
26526992322,1cyclictest5409-21perf10:00:010
26526992322,1cyclictest28315-21diskmemload11:55:200
26526992322,1cyclictest18454-21kworker/u8:009:22:490
26526992322,1cyclictest18186-21diskmemload10:47:340
26526992321,2cyclictest15763-21sh10:39:590
2652699230,23cyclictest18186-21diskmemload12:11:030
2652699230,23cyclictest0-21swapper/012:32:180
2652699230,23cyclictest0-21swapper/011:41:590
2652699230,23cyclictest0-21swapper/011:17:200
2652699230,22cyclictest3890-21turbostat11:00:000
2652699230,22cyclictest18186-21diskmemload09:16:280
2652699230,22cyclictest16494-21ssh10:40:450
2652699230,22cyclictest0-21swapper/011:36:000
2652699230,22cyclictest0-21swapper/007:55:000
2652699230,1cyclictest313-21systemd-journal08:25:000
2652699230,1cyclictest0-21swapper/012:00:100
2652699230,1cyclictest0-21swapper/011:51:100
2652699230,0cyclictest0-21swapper/011:21:040
26533992222,0cyclictest0-21swapper/311:47:263
26533992222,0cyclictest0-21swapper/311:30:463
26533992222,0cyclictest0-21swapper/311:10:503
26533992222,0cyclictest0-21swapper/310:01:443
26533992222,0cyclictest0-21swapper/309:41:013
26533992221,1cyclictest28642-21apt-get11:55:563
26533992220,1cyclictest1301-21latency_hist07:25:013
2653399220,2cyclictest4188-21ssh12:35:023
2653399220,2cyclictest30895-21sh10:22:363
2653399220,22cyclictest0-21swapper/311:26:353
2653399220,22cyclictest0-21swapper/309:15:063
2653399220,21cyclictest13744-21apt-get09:00:563
2653399220,21cyclictest0-21swapper/310:40:563
2653399220,21cyclictest0-21swapper/310:16:043
2653399220,1cyclictest6954-21df12:05:583
2653399220,1cyclictest28362-21apt-get09:50:563
2653399220,1cyclictest0-21swapper/312:33:043
2653399220,1cyclictest0-21swapper/312:00:383
2653399220,0cyclictest0-21swapper/310:05:583
2653399220,0cyclictest0-21swapper/309:25:343
26531992222,0cyclictest18186-21diskmemload09:36:352
26531992222,0cyclictest0-21swapper/212:11:292
26531992222,0cyclictest0-21swapper/211:30:132
26531992222,0cyclictest0-21swapper/210:57:462
26531992222,0cyclictest0-21swapper/210:16:412
26531992221,1cyclictest11275-21apt-get08:55:562
26531992220,1cyclictest31315-21perf07:20:012
26531992220,1cyclictest3068-21cron08:35:002
26531992220,1cyclictest18220-21munin-run09:10:012
26531992220,1cyclictest16514-21ssh10:40:482
2653199220,21cyclictest8076-21sed12:06:062
2653199220,21cyclictest5529-21lockfile-remove08:40:002
2653199220,21cyclictest29215-21ssh09:20:172
2653199220,21cyclictest28257-21ssh11:23:012
2653199220,21cyclictest18467-21ssh09:10:042
2653199220,21cyclictest0-21swapper/211:50:292
2653199220,21cyclictest0-21swapper/208:15:562
2653199220,1cyclictest770-21ssh10:25:552
2653199220,1cyclictest29078-21sh12:26:302
2653199220,1cyclictest15824-21ssh11:42:092
2653199220,0cyclictest0-21swapper/211:03:492
2653199220,0cyclictest0-21swapper/209:28:412
26527992222,0cyclictest0-21swapper/109:16:501
26527992221,1cyclictest18186-21diskmemload12:02:461
26527992220,1cyclictest8628-21awk07:40:001
26527992220,1cyclictest26180-21apt-get08:15:561
26527992220,1cyclictest0-21swapper/110:40:181
2652799220,22cyclictest0-21swapper/110:26:081
2652799220,21cyclictest0-21swapper/109:35:421
2652799220,21cyclictest0-21swapper/107:25:561
2652799220,1cyclictest30688-21munin-run08:25:001
2652799220,1cyclictest256752chrt11:51:311
2652799220,1cyclictest18318-21ssh11:45:481
2652799220,1cyclictest0-21swapper/111:00:581
26526992222,0cyclictest0-21swapper/009:35:210
26526992221,1cyclictest0-21swapper/010:50:380
26526992221,0cyclictest0-21swapper/010:07:020
26526992220,1cyclictest28286-21awk08:20:010
26526992220,1cyclictest19812-21sh12:17:570
26526992220,1cyclictest0-21swapper/012:25:130
2652699220,22cyclictest0-21swapper/009:46:150
2652699220,21cyclictest0-21swapper/012:05:170
2652699220,21cyclictest0-21swapper/011:48:070
2652699220,21cyclictest0-21swapper/010:01:170
2652699220,21cyclictest0-21swapper/009:30:020
2652699220,21cyclictest0-21swapper/008:30:560
2652699220,1cyclictest0-21swapper/012:38:210
2652699220,1cyclictest0-21swapper/008:05:010
2652699220,0cyclictest18186-21diskmemload10:15:510
2652699220,0cyclictest0-21swapper/011:30:150
26533992121,0cyclictest0-21swapper/310:46:263
26533992121,0cyclictest0-21swapper/310:33:293
2653399210,21cyclictest0-21swapper/309:20:343
2653399210,20cyclictest0-21swapper/309:10:563
2653399210,20cyclictest0-21swapper/307:35:563
2653399210,1cyclictest25866-21perf08:15:013
2653399210,1cyclictest0-21swapper/311:20:273
26531992121,0cyclictest0-21swapper/212:18:362
26531992120,1cyclictest28906-21ls07:15:012
26531992120,1cyclictest20535-21ssh11:46:362
2653199210,0cyclictest1024-21apt-get09:55:562
2653199210,0cyclictest0-21swapper/211:05:292
2653199210,0cyclictest0-21swapper/209:00:562
26527992121,0cyclictest0-21swapper/112:16:071
26527992120,1cyclictest28341-21kworker/1:007:45:551
26527992119,1cyclictest0-21swapper/107:25:011
26526992121,0cyclictest0-21swapper/008:36:000
26526992120,1cyclictest23787-21apt-get08:10:560
26526992119,1cyclictest615-21awk08:30:000
2652699210,21cyclictest0-21swapper/009:50:090
2652699210,21cyclictest0-21swapper/007:30:540
26533992020,0cyclictest4033-21apt-get07:30:553
26533992020,0cyclictest0-21swapper/308:20:553
2653399200,0cyclictest5843-21apt-get08:40:563
26531992020,0cyclictest31652-21apt-get07:20:562
26531992020,0cyclictest0-21swapper/207:50:542
26531992018,1cyclictest23477-21munin-run08:10:012
2653199200,20cyclictest0-21swapper/208:25:562
26527992020,0cyclictest0-21swapper/107:55:561
26527992018,1cyclictest622-21gdbus08:10:011
26526992019,0cyclictest0-21swapper/008:40:570
26526992018,1cyclictest3717-21awk07:30:010
26533991919,0cyclictest18999-21apt-get08:00:563
26533991919,0cyclictest16189-21apt-get07:55:563
26533991919,0cyclictest0-21swapper/308:15:563
26533991918,0cyclictest0-21swapper/308:05:543
26531991919,0cyclictest0-21swapper/208:40:572
26531991919,0cyclictest0-21swapper/208:00:022
26531991919,0cyclictest0-21swapper/207:55:562
26527991919,0cyclictest0-21swapper/108:50:531
26527991917,1cyclictest28920-21munin-run07:15:011
2652799190,18cyclictest14492-21munin-node08:40:481
26526991919,0cyclictest0-21swapper/007:20:560
26533991818,0cyclictest0-21swapper/307:25:563
26531991818,0cyclictest23787-21apt-get08:10:552
26531991818,0cyclictest0-21swapper/207:35:562
26531991817,0cyclictest28608-21apt-get08:20:552
26531991816,1cyclictest8543-21dump-pmu-power08:50:002
2653199180,1cyclictest3714-21munin-run07:30:012
26527991818,0cyclictest18271-21apt-get08:00:021
26527991815,2cyclictest5518-21perf08:40:001
26526991818,0cyclictest0-21swapper/009:05:550
26526991814,4cyclictest8599-21latency_hist07:40:000
26533991717,0cyclictest0-21swapper/308:50:533
26533991716,1cyclictest0-21swapper/307:45:003
26527991716,1cyclictest976-21apt-get08:30:561
26533991616,0cyclictest13792-21apt-get07:50:543
26533991616,0cyclictest0-21swapper/308:25:563
26533991616,0cyclictest0-21swapper/307:10:553
26533991615,1cyclictest3420-21apt-get08:35:553
26531991616,0cyclictest0-21swapper/208:50:532
26531991616,0cyclictest0-21swapper/207:45:562
2652799163,6cyclictest151rcu_preempt07:50:591
26527991616,0cyclictest0-21swapper/108:55:561
26527991614,1cyclictest0-21swapper/108:26:031
26526991616,0cyclictest8876-21apt-get08:50:530
26526991616,0cyclictest0-21swapper/009:00:560
26526991616,0cyclictest0-21swapper/008:50:010
2652699160,0cyclictest0-21swapper/007:40:540
2653199150,1cyclictest9995-21ntp_states07:41:002
26527991515,0cyclictest0-21swapper/109:05:551
26526991515,0cyclictest0-21swapper/007:15:550
26526991513,1cyclictest28886-21idleruntime-cro07:15:010
26526991513,1cyclictest13402-21sendmail-msp09:00:010
2653399140,14cyclictest0-21swapper/308:45:243
2653399140,14cyclictest0-21swapper/307:45:043
2653399140,14cyclictest0-21swapper/307:16:413
2653399140,0cyclictest0-21swapper/309:05:263
2653399140,0cyclictest0-21swapper/308:56:123
2653399140,0cyclictest0-21swapper/308:30:433
2653199140,14cyclictest0-21swapper/207:30:292
26527991414,0cyclictest0-21swapper/108:45:521
2652799140,14cyclictest0-21swapper/109:00:211
2652799140,14cyclictest0-21swapper/107:16:081
2652799140,0cyclictest0-21swapper/107:30:451
2652699140,14cyclictest0-21swapper/008:05:330
2652699140,14cyclictest0-21swapper/007:55:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional