You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 15:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Aug 13, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1560929581,9sleep00-21swapper/007:09:190
1557629077,9sleep20-21swapper/207:08:532
1558228269,9sleep30-21swapper/307:08:583
1549328063,13sleep10-21swapper/107:07:481
209712750,0sleep30-21swapper/311:30:303
1575499301,3cyclictest3314-21tr09:25:153
1575499272,3cyclictest5418-21apt-get10:05:133
1575099273,23cyclictest1142-21runrttasks11:09:142
1575099270,4cyclictest0-21swapper/210:05:132
1575499253,4cyclictest21012-21apt-get09:55:133
1575099250,6cyclictest0-21swapper/212:14:562
1575499248,15cyclictest16549-21cstates10:50:153
1575499248,15cyclictest16549-21cstates10:50:143
1575499234,3cyclictest23447-21apt-get10:35:123
1575499231,5cyclictest0-21swapper/310:40:203
1575499230,3cyclictest9409-21diskmemload12:20:223
1574299230,13cyclictest0-21swapper/008:00:150
1575499225,3cyclictest9409-21diskmemload09:31:003
1575499221,5cyclictest13517-21ssh11:45:203
1574899223,1cyclictest9409-21diskmemload12:26:021
15748992219,2cyclictest0-21swapper/107:25:131
1574899221,2cyclictest0-21swapper/111:51:531
1574299220,4cyclictest0-21swapper/011:06:080
1574299220,16cyclictest0-21swapper/009:05:130
1574299220,16cyclictest0-21swapper/009:05:120
1575499213,3cyclictest0-21swapper/311:38:323
1575499210,3cyclictest0-21swapper/311:13:543
1575099214,15cyclictest30671-21cut10:20:142
1575099212,1cyclictest27357-21sh09:20:152
15750992112,8cyclictest5609-21tr09:45:222
1575099210,4cyclictest0-21swapper/209:31:092
1575099210,3cyclictest0-21swapper/212:25:442
1575099210,3cyclictest0-21swapper/210:25:542
1575099210,2cyclictest0-21swapper/210:47:352
1575099210,20cyclictest0-21swapper/210:15:132
1574899212,1cyclictest141rcu_preempt10:52:151
1574899212,1cyclictest141rcu_preempt10:52:141
15748992120,1cyclictest0-21swapper/110:45:431
1574899210,21cyclictest0-21swapper/110:28:251
1575499200,19cyclictest0-21swapper/312:28:543
1575099201,2cyclictest0-21swapper/209:55:132
1575099200,2cyclictest0-21swapper/210:01:132
1575099200,1cyclictest0-21swapper/210:36:182
1575099200,1cyclictest0-21swapper/209:36:412
15748992018,1cyclictest0-21swapper/107:15:131
1574899200,1cyclictest0-21swapper/111:41:411
1575499192,2cyclictest30670-21grep10:20:143
1575499190,1cyclictest0-21swapper/309:45:123
1575099193,2cyclictest18642-21aten2_r7power_e09:15:142
1575099193,16cyclictest9387-21kworker/u8:009:10:092
15748991919,0cyclictest0-21swapper/112:32:351
1574299190,3cyclictest0-21swapper/009:57:300
1574299190,1cyclictest0-21swapper/011:53:460
235202180,0sleep10-21swapper/109:36:441
1575499183,3cyclictest30738-21sh10:58:383
1575499180,17cyclictest0-21swapper/311:03:523
1575099183,3cyclictest0-21swapper/208:45:102
1575099181,3cyclictest1832-21sh11:20:052
1575099180,17cyclictest0-21swapper/211:03:522
1574299187,5cyclictest31347-21sendmail10:40:010
1575499172,3cyclictest0-21swapper/309:24:333
1575099175,9cyclictest0-21swapper/211:50:542
1575099171,4cyclictest11891-21/usr/sbin/munin11:45:142
1575099171,3cyclictest0-21swapper/212:05:352
1574299176,6cyclictest21102-21munin-run12:10:000
1574299175,6cyclictest2064-21munin-run08:55:000
1575499169,6cyclictest27256-21grep07:30:233
1575499168,7cyclictest18327-21cut08:20:203
1575499164,2cyclictest32483-21ssh11:56:433
1575499163,1cyclictest9409-21diskmemload12:08:203
1575499162,3cyclictest9409-21diskmemload09:38:203
1575499161,5cyclictest10410-21proc_pri11:05:213
1575499161,1cyclictest0-21swapper/310:28:223
15754991611,4cyclictest31414-21pmu-power12:15:243
1575099169,6cyclictest17895-21tr11:10:132
1575099169,2cyclictest225-21systemd-journal08:19:592
1575099168,7cyclictest7160-21sh12:00:422
1575099161,2cyclictest11408-21sh12:22:552
1575099161,2cyclictest0-21swapper/210:54:062
1575099161,2cyclictest0-21swapper/210:54:052
15750991611,3cyclictest5463-21cron12:20:002
1574899169,6cyclictest14340-21cat09:50:251
15748991616,0cyclictest0-21swapper/112:35:261
15748991616,0cyclictest0-21swapper/112:35:251
15748991616,0cyclictest0-21swapper/111:26:421
1574899160,15cyclictest0-21swapper/109:41:181
1574299161,11cyclictest11733-21ssh11:45:030
1575499159,5cyclictest23464-21grep10:15:213
1575499158,6cyclictest5699-21sed09:00:183
1575499158,6cyclictest31032-21sh11:17:463
1575499158,6cyclictest23145-21ssh11:50:593
1575499158,6cyclictest2028-21chrt08:54:513
1575499158,6cyclictest17420-21ssh10:30:463
1575499158,6cyclictest13607-21ssh11:26:223
1575499157,7cyclictest6790-21sh12:00:273
1575499157,7cyclictest32302-21cut12:35:183
1575499157,7cyclictest32302-21cut12:35:173
1575499157,7cyclictest23194-21grep12:10:243
1575499153,3cyclictest9409-21diskmemload11:20:203
1575499151,2cyclictest0-21swapper/311:45:003
1575499150,14cyclictest31975-21cut07:40:223
1575499150,14cyclictest24269-21sed12:30:193
1575499150,14cyclictest0-21swapper/310:14:173
1575099159,5cyclictest1755-21wc07:45:192
1575099158,6cyclictest6183-21sh10:44:092
1575099158,6cyclictest3308-21sh11:40:082
1575099158,6cyclictest30799-21cut07:40:142
1575099158,6cyclictest24080-21sed08:30:262
1575099157,7cyclictest6884-21perf12:40:012
1575099157,7cyclictest6884-21perf12:40:002
1575099151,5cyclictest2698-21sh09:44:142
15750991510,4cyclictest3050-21head08:55:172
1575099150,14cyclictest24708-21sed12:30:242
1575099150,14cyclictest197572sleep210:32:322
1574899158,4cyclictest0-21swapper/110:00:001
1574899153,10cyclictest14258-21runrttasks12:05:201
1574899151,13cyclictest9409-21diskmemload10:31:141
1574299151,8cyclictest7595-21munin-run07:59:590
1574299150,4cyclictest0-21swapper/010:14:120
1575499149,4cyclictest22144-21cron08:30:003
1575499148,5cyclictest29619-21cut10:00:143
1575499148,5cyclictest29158-21chrt08:43:213
1575499148,5cyclictest17874-21chrt07:12:223
1575499148,5cyclictest17874-21chrt07:12:213
1575499148,5cyclictest17521-21sh09:14:353
1575499147,6cyclictest5706-21tr07:55:143
1575499147,6cyclictest3522-21cut08:55:213
1575499147,6cyclictest3054-21taskset07:50:023
1575499147,6cyclictest14191-21ssh09:50:243
1575499147,2cyclictest9862-21ssh10:45:293
1575499147,2cyclictest15086-21cut08:15:133
1575499143,9cyclictest29126-21sed09:40:183
1575499142,2cyclictest225-21systemd-journal08:37:583
1575499140,4cyclictest225-21systemd-journal08:05:003
1575099149,4cyclictest27204-21cut08:40:122
1575099149,4cyclictest15769-21grep10:10:272
1575099148,5cyclictest9128-21chrt09:06:332
1575099148,5cyclictest9128-21chrt09:06:322
1575099148,5cyclictest23327-21taskset07:24:492
1575099147,6cyclictest6820-21chrt09:02:242
1575099147,6cyclictest6351-21ssh09:26:282
1575099147,6cyclictest28010-21head11:15:262
1575099147,6cyclictest27122-21date08:40:012
1575099147,2cyclictest16321-21ssh09:51:472
1575099144,4cyclictest475-21in:imuxsock07:30:012
1575099143,2cyclictest11774-21ssh11:25:222
1575099142,2cyclictest225-21systemd-journal07:20:012
15750991411,2cyclictest13566-21cut08:10:192
1575099140,2cyclictest0-21swapper/210:56:572
1575099140,13cyclictest32456-21chrt11:37:552
1574899149,4cyclictest3690-21sort09:25:161
15748991411,2cyclictest29523-21wc08:45:001
1574299149,1cyclictest13-21ksoftirqd/011:58:450
1574299146,2cyclictest225-21systemd-journal11:43:330
1574299143,1cyclictest9409-21diskmemload09:23:240
15742991412,1cyclictest9409-21diskmemload10:04:520
15742991410,3cyclictest3464-21uname11:20:220
15742991410,3cyclictest2063-21wc11:00:220
1575499139,2cyclictest7437-21grep09:05:143
1575499139,2cyclictest7437-21grep09:05:133
1575499137,1cyclictest30228-21taskset07:36:513
1575499136,2cyclictest23202-21ssh09:17:273
1575499136,2cyclictest12055-21taskset08:06:193
15754991310,2cyclictest14719-21cron08:15:003
1575499130,4cyclictest225-21systemd-journal07:50:013
1575099137,5cyclictest26840-21if_enp0s31f607:30:182
1575099135,2cyclictest30725-21grep11:55:262
15750991310,2cyclictest3801-21grep07:50:172
15750991310,2cyclictest21002-21wc08:25:182
15750991310,2cyclictest19022-21grep08:20:272
1575099130,12cyclictest6246-21sed07:55:152
1574899139,3cyclictest17680-21grep08:20:141
1574899139,2cyclictest22485-21grep08:30:131
1574899133,6cyclictest9901-21apt11:25:001
15748991311,1cyclictest9409-21diskmemload11:07:181
15748991310,2cyclictest31519-21wc12:15:241
15748991310,2cyclictest15592-21wc10:10:271
15748991310,2cyclictest12904-21grep08:10:151
15748991310,2cyclictest12695-21cut11:45:141
1574299139,3cyclictest26068-21grep07:30:140
1574299139,3cyclictest20492-21grep09:15:260
1574299132,6cyclictest1508-21munin-run11:20:010
15742991310,2cyclictest8245-21sh10:45:150
15742991310,2cyclictest7672-21grep12:20:220
15742991310,2cyclictest4032-21grep08:55:250
15742991310,2cyclictest32148-21cut12:35:180
15742991310,2cyclictest32148-21cut12:35:180
15742991310,2cyclictest30053-21grep12:15:150
15742991310,2cyclictest29886-21wc07:35:260
15742991310,2cyclictest23373-21cat08:30:180
15742991310,2cyclictest15464-21head12:25:160
15742991310,2cyclictest14913-21wc08:15:000
15742991310,2cyclictest12419-21cat08:09:590
15742991310,2cyclictest12041-21grep09:10:260
1574299130,12cyclictest12865-21wc09:30:210
1575499129,2cyclictest24288-21grep07:25:163
1575499126,2cyclictest245172sleep308:34:593
1575499126,1cyclictest30269-21sed08:45:133
1575499121,4cyclictest21847-21cut07:20:173
1575099127,4cyclictest97452sleep208:02:102
1575099126,2cyclictest16747-21cut07:10:162
1575099126,2cyclictest16747-21cut07:10:152
1575099122,3cyclictest28445-21cut07:35:132
1575099122,2cyclictest225-21systemd-journal08:09:592
1575099120,5cyclictest31875-21cron08:50:012
1575099120,5cyclictest20210-21ls11:30:212
1574899129,2cyclictest4972-21basename09:00:141
1574899129,2cyclictest32415-21grep07:40:281
1574899129,2cyclictest27654-21ssh11:15:231
1574899129,2cyclictest24799-21ls10:35:211
1574899129,2cyclictest21744-21sed08:25:251
1574899129,2cyclictest19670-21grep11:30:161
15748991211,1cyclictest0-21swapper/111:01:581
15748991211,1cyclictest0-21swapper/110:58:051
15748991211,1cyclictest0-21swapper/110:40:141
1574899121,10cyclictest0-21swapper/108:55:121
15748991210,1cyclictest10445-21grep09:10:141
1574899120,12cyclictest0-21swapper/109:48:421
1574899120,10cyclictest979-21missed_timers08:50:181
1574299129,2cyclictest5635-21grep09:00:170
1574299129,2cyclictest4873-21cut09:45:160
1574299129,2cyclictest31759-21cat07:40:200
1574299129,2cyclictest25554-21sleep11:15:000
1574299129,2cyclictest24892-21grep10:55:150
1574299129,2cyclictest21499-21grep07:20:140
1574299129,2cyclictest18813-21cat08:20:250
1574299129,2cyclictest18276-21idleruntime-cro11:29:590
1574299129,2cyclictest15589-21grep08:15:160
1574299129,2cyclictest12663-21munin-plugin-st12:04:590
1574299128,3cyclictest30955-21ls10:20:150
1574299122,1cyclictest141rcu_preempt10:52:420
1574299122,1cyclictest141rcu_preempt10:52:410
15742991210,1cyclictest20745-21grep08:25:160
1575499118,2cyclictest18860-21tr07:15:133
1574899118,3cyclictest29387-21ssh11:35:321
1574899118,2cyclictest7018-21fw_packets12:20:171
1574899118,2cyclictest6003-21df10:05:161
1574899118,2cyclictest5114-21grep12:00:141
1574899118,2cyclictest4327-21cat12:00:001
1574899118,2cyclictest30263-21df08:45:131
1574899118,2cyclictest29153-21sed07:35:171
1574899118,2cyclictest26450-21ssh10:17:101
1574899118,2cyclictest25-21ksoftirqd/109:20:161
1574899118,2cyclictest2409-21ssh10:03:051
1574899118,2cyclictest23436-21ssh09:17:381
1574899118,2cyclictest19490-21ls11:10:251
1574899118,2cyclictest10964-21grep08:05:141
1574899117,3cyclictest7125-21grep07:55:241
1574899117,1cyclictest32650-21ssh10:20:271
15748991110,1cyclictest0-21swapper/109:33:461
1574299119,1cyclictest32539-21grep10:40:160
1574299118,2cyclictest5306-21ssh10:05:090
1574299118,2cyclictest30211-21expr08:45:120
1574299118,2cyclictest29585-21ssh10:19:360
1574299118,2cyclictest28946-21taskset08:40:370
1574299118,2cyclictest27748-21ssh12:32:320
1574299118,2cyclictest27505-21grep11:35:130
1574299118,2cyclictest25111-21ssh11:33:450
1574299118,2cyclictest18858-21grep07:15:140
1574299118,2cyclictest17180-21ssh10:30:350
1574299118,2cyclictest1321-21cut07:45:150
1574299117,3cyclictest3849-21grep09:25:160
1574299117,3cyclictest20148-21tr09:35:140
15742991110,1cyclictest0-21swapper/007:25:130
1574299110,1cyclictest0-21swapper/009:41:570
1574899108,1cyclictest8421-21grep09:05:191
1574899108,1cyclictest8421-21grep09:05:181
1574899108,1cyclictest26401-21cat07:30:151
1574899108,1cyclictest26386-21date08:35:251
1574899108,1cyclictest21198-21appstreamcli07:20:001
1574899108,1cyclictest17123-21ntpq07:10:191
1574899108,1cyclictest17123-21ntpq07:10:181
1574899107,2cyclictest935-21tr07:45:131
1574899107,2cyclictest8119-21grep08:00:161
1574899107,2cyclictest3872-21grep07:50:181
1574899107,2cyclictest21391-21ssh12:10:041
1574899107,2cyclictest15658-21cut08:15:161
1574299108,1cyclictest247592sleep008:35:090
1574299108,1cyclictest21353-21ssh12:10:020
1574299107,2cyclictest6993-21expr10:25:140
1574299107,2cyclictest4922-21chrt07:51:300
1574299107,2cyclictest12418-21ssh09:50:060
1574299100,9cyclictest17916-21taskset07:12:540
1574299100,9cyclictest17916-21taskset07:12:540
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional