You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 04:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Wed Sep 16, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
244042183107,28sleep10-21swapper/107:09:391
243562170103,29sleep00-21swapper/007:09:020
244052165115,27sleep20-21swapper/207:09:412
547521630,1sleep00-21swapper/012:16:040
241162162111,28sleep30-21swapper/307:05:563
244482790,2sleep10-21swapper/110:30:221
34182730,3sleep30-21swapper/312:15:143
48162680,5sleep1141rcu_preempt09:28:311
231572610,16sleep20-21swapper/211:41:262
33912590,2sleep20-21swapper/210:15:152
208062590,2sleep20-21swapper/212:04:252
318862580,1sleep00-21swapper/012:11:500
318862580,1sleep00-21swapper/012:11:500
321402560,1sleep00-21swapper/009:48:390
115002560,4sleep0131rcuc/010:45:140
183152540,10sleep012-21ksoftirqd/009:38:360
90212520,2sleep10-21swapper/109:30:591
307602500,2sleep00-21swapper/011:47:190
268562490,2chrt0-21swapper/110:08:351
210442490,3chrt0-21swapper/310:04:323
48962480,0sleep1211rcuc/111:28:251
160442470,9sleep228-21ksoftirqd/211:36:042
101972450,1sleep10-21swapper/109:55:271
198512440,1sleep20-21swapper/210:26:592
23592430,2sleep20-21swapper/210:14:462
299252410,2sleep20-21swapper/209:22:532
214232410,1sleep021420-21awk12:05:000
315312380,3chrt0-21swapper/312:11:303
315312380,3chrt0-21swapper/312:11:293
2470699313,10cyclictest34-21ksoftirqd/312:05:363
214352310,2chrt0-21swapper/009:16:090
77902300,2sleep00-21swapper/010:42:050
256252290,6sleep2271rcuc/210:30:592
163872290,2sleep10-21swapper/111:12:401
24706992817,6cyclictest11859-21diskmemload09:55:253
24706992811,5cyclictest34-21ksoftirqd/311:20:183
24706992811,4cyclictest34-21ksoftirqd/309:16:373
24706992810,5cyclictest34-21ksoftirqd/312:03:053
24704992817,7cyclictest0-21swapper/112:10:031
24704992817,7cyclictest0-21swapper/112:10:031
2470699279,9cyclictest34-21ksoftirqd/311:56:283
2470699274,11cyclictest34-21ksoftirqd/310:22:523
24706992711,5cyclictest34-21ksoftirqd/310:08:363
2470399271,24cyclictest0-21swapper/007:18:120
176622270,5sleep122-21ksoftirqd/110:50:001
113322270,3sleep10-21swapper/111:33:061
304992260,2chrt0-21swapper/310:59:333
267752260,2sleep30-21swapper/310:32:093
2470699268,7cyclictest141rcu_preempt11:01:393
2470699268,10cyclictest34-21ksoftirqd/312:21:543
2470699266,4cyclictest34-21ksoftirqd/312:32:533
2470699264,12cyclictest34-21ksoftirqd/311:05:173
2470699263,12cyclictest34-21ksoftirqd/309:30:163
24706992619,5cyclictest34-21ksoftirqd/308:00:003
2470699261,23cyclictest0-21swapper/307:18:403
24706992611,4cyclictest34-21ksoftirqd/310:46:553
24706992610,4cyclictest34-21ksoftirqd/311:46:043
24706992610,2cyclictest141rcu_preempt09:53:343
2470599261,23cyclictest0-21swapper/207:20:272
2470599261,23cyclictest0-21swapper/207:18:482
2470499262,17cyclictest30095-21ssh10:35:121
2470499261,24cyclictest0-21swapper/107:20:351
2470499261,23cyclictest0-21swapper/107:34:271
2470499261,1cyclictest0-21swapper/107:19:241
2470499260,24cyclictest0-21swapper/108:18:351
24703992624,1cyclictest0-21swapper/007:20:390
31282250,2sleep00-21swapper/012:39:070
264322250,1sleep00-21swapper/010:08:110
2470699259,4cyclictest34-21ksoftirqd/311:53:083
2470699259,3cyclictest141rcu_preempt09:40:233
2470699259,2cyclictest141rcu_preempt12:35:093
2470699258,5cyclictest34-21ksoftirqd/311:40:073
2470699258,3cyclictest141rcu_preempt10:38:353
2470699256,12cyclictest34-21ksoftirqd/311:25:123
2470699251,11cyclictest34-21ksoftirqd/311:17:523
2470499252,16cyclictest16582-21ssh12:25:111
2470699249,4cyclictest34-21ksoftirqd/311:33:063
2470699249,4cyclictest34-21ksoftirqd/310:54:053
2470699247,9cyclictest34-21ksoftirqd/308:32:253
2470699247,4cyclictest34-21ksoftirqd/309:36:433
2470699244,11cyclictest34-21ksoftirqd/309:25:053
24706992422,1cyclictest0-21swapper/307:34:273
2470699241,1cyclictest0-21swapper/307:20:273
2470699241,12cyclictest34-21ksoftirqd/310:15:023
2470699240,22cyclictest0-21swapper/308:07:303
2470699240,12cyclictest34-21ksoftirqd/309:21:313
24705992422,1cyclictest0-21swapper/207:34:192
2470599242,11cyclictest9826-21ssh10:20:122
24704992422,1cyclictest0-21swapper/108:09:091
24703992417,5cyclictest18289-21sort10:50:140
220932240,2sleep10-21swapper/110:52:531
197072240,2sleep20-21swapper/210:50:292
2470699239,4cyclictest34-21ksoftirqd/310:11:323
2470699239,3cyclictest141rcu_preempt11:37:273
2470699238,4cyclictest34-21ksoftirqd/310:26:163
2470699238,4cyclictest34-21ksoftirqd/308:35:263
2470699236,4cyclictest34-21ksoftirqd/310:40:313
2470699235,7cyclictest34-21ksoftirqd/308:00:243
2470699235,2cyclictest141rcu_preempt08:59:593
2470699233,9cyclictest34-21ksoftirqd/309:00:163
2470699232,11cyclictest34-21ksoftirqd/309:11:113
2470699231,11cyclictest34-21ksoftirqd/309:49:433
2470599233,7cyclictest21927-21ssh11:16:292
2470499231,15cyclictest8252-21sh11:55:041
158252230,2sleep20-21swapper/209:12:132
45252220,1sleep00-21swapper/009:28:130
275032220,1chrt0-21swapper/008:30:100
252852220,2sleep0131rcuc/007:10:170
2470699227,8cyclictest34-21ksoftirqd/308:41:013
2470699227,4cyclictest34-21ksoftirqd/308:15:123
2470699226,5cyclictest34-21ksoftirqd/312:29:553
2470699225,9cyclictest34-21ksoftirqd/311:12:523
24706992218,3cyclictest34-21ksoftirqd/307:45:003
24706992215,5cyclictest34-21ksoftirqd/307:50:003
24706992212,7cyclictest19750-21awk08:10:223
24706992210,4cyclictest34-21ksoftirqd/308:50:263
2470599222,5cyclictest0-21swapper/211:28:222
24705992214,6cyclictest26185-21smartctl08:25:232
24705992214,6cyclictest13252-21ls11:10:192
24705992212,7cyclictest10942-21open_inodes07:50:192
24705992211,8cyclictest27532-21df11:45:142
2470499224,8cyclictest15367-21unixbench_singl11:35:281
2470499222,14cyclictest23330-21ssh12:30:131
24704992213,7cyclictest23170-21python312:05:231
24704992211,5cyclictest29795-21seq09:46:201
2470499220,8cyclictest11859-21diskmemload09:10:231
2470399229,3cyclictest105242cut07:50:130
2470399222,14cyclictest28692-21chrt10:10:100
24703992214,6cyclictest6906-21ls08:55:230
2470699218,4cyclictest34-21ksoftirqd/308:20:143
2470599219,9cyclictest28-21ksoftirqd/208:15:122
2470599212,13cyclictest9865-21sh12:20:122
2470599211,5cyclictest5975-21ssh09:29:402
24705992113,6cyclictest7901-21ls09:55:012
24705992113,6cyclictest278-21systemd-journal07:50:002
24705992113,6cyclictest13974-21fw_packets09:35:162
24705992112,6cyclictest24899-21cat10:55:132
24705992111,8cyclictest28284-21latency_hist10:10:002
2470599211,17cyclictest14633-21df_inode08:00:162
2470499213,15cyclictest31723-21grep12:35:251
24704992114,4cyclictest5299-21ls11:05:001
24704992113,6cyclictest491-21seq07:28:351
2470499211,13cyclictest20436-21ssh11:40:131
24704992110,7cyclictest52850irq/140-enp3s0-10:10:121
24703992113,6cyclictest7877-21pluginstate09:30:240
24703992113,6cyclictest22372-21seq08:17:110
24703992113,6cyclictest12802-21entropy11:10:150
24703992112,6cyclictest8817-21pluginstate07:45:200
2470399210,19cyclictest3670-21ssh11:03:240
2470699208,4cyclictest34-21ksoftirqd/308:46:293
2470699201,9cyclictest34-21ksoftirqd/309:05:183
2470699201,9cyclictest34-21ksoftirqd/309:05:183
2470699201,9cyclictest34-21ksoftirqd/307:29:213
24706992015,4cyclictest34-21ksoftirqd/307:55:003
24706992012,5cyclictest26673-21seq08:27:053
2470599202,13cyclictest30117-21ssh12:35:132
2470599201,4cyclictest0-21swapper/211:24:182
24705992014,4cyclictest9309-21sh09:31:162
24705992014,4cyclictest15027-21sh10:00:012
24705992014,4cyclictest13186-21sh11:34:582
24705992013,5cyclictest876-21seq12:13:142
24705992013,5cyclictest876-21seq12:13:142
24705992013,5cyclictest23030-21cut12:30:012
24705992013,4cyclictest26803-21sh09:45:062
24705992012,6cyclictest6571-21memory11:05:192
24705992011,7cyclictest31355-21cat07:25:142
24705992011,7cyclictest29570-21awk08:35:152
24705992011,6cyclictest278-21systemd-journal09:45:002
2470499202,12cyclictest22882-21apt-get08:20:151
2470499201,6cyclictest0-21swapper/112:15:211
24704992014,4cyclictest11484-21unixbench_multi12:20:261
24704992012,6cyclictest17963-21sed10:25:231
24704992012,5cyclictest29278-21basename08:35:011
24704992011,7cyclictest20337-21python309:15:231
2470399209,8cyclictest16551-21cut08:05:130
24703992014,4cyclictest4102-21grep07:35:200
24703992014,4cyclictest31655-21sh09:24:380
24703992013,5cyclictest23642-21expr12:30:130
24703992013,4cyclictest17856-21sh09:14:170
24703992012,6cyclictest363-21seq07:26:590
24703992012,6cyclictest25036-21diskstats10:55:140
24703992012,6cyclictest19520-21open_inodes08:10:200
24703992012,6cyclictest15858-21seq08:00:530
24703992011,5cyclictest25918-21ssh11:20:120
49012190,3chrt0-21swapper/109:52:051
325932190,2chrt0-21swapper/210:36:142
247222190,2sleep30-21swapper/307:10:143
24705991913,4cyclictest26837-21sh12:32:112
24705991913,4cyclictest22736-21missed_timers12:05:202
24705991911,6cyclictest28131-21memory08:30:162
24705991911,6cyclictest24747-21apt-get07:10:132
24705991911,5cyclictest11859-21diskmemload10:40:092
24705991910,7cyclictest7997-21cat09:00:142
24705991910,6cyclictest9034-21idleruntime11:55:162
24705991910,6cyclictest5881-21expr08:55:132
24705991910,6cyclictest1434-21acpi11:50:102
24705991910,5cyclictest52950irq/141-enp3s0-12:15:132
2470499192,12cyclictest25727-21apt-get09:20:121
2470499192,11cyclictest15199-21apt-get10:00:121
24704991912,5cyclictest2265-21sed11:50:181
24704991912,5cyclictest20149-21ntp_kernel_err11:15:201
24704991911,6cyclictest8780-21seq10:19:251
24704991911,6cyclictest30999-21wc11:00:011
24704991911,6cyclictest19184-21if_enp3s008:10:161
24704991910,6cyclictest28827-21grep11:45:251
2470399199,7cyclictest24988-21seq08:25:020
24703991917,1cyclictest0-21swapper/007:34:190
24703991914,3cyclictest1546-21cpuspeed_turbos09:50:110
24703991914,3cyclictest13746-21sh12:22:350
24703991913,4cyclictest10877-21sh09:56:070
24703991912,4cyclictest11000-21sh10:20:220
24703991911,6cyclictest7235-21expr11:30:130
24703991911,6cyclictest23876-21awk08:20:220
24703991911,6cyclictest2214-21sed08:45:190
24703991911,6cyclictest17582-21ntp_kernel_err12:25:200
2470399191,13cyclictest15216-21ssh10:00:110
24703991910,6cyclictest1543-21ls11:25:230
211642190,2chrt0-21swapper/010:28:160
2470699181,7cyclictest0-21swapper/307:35:263
2470599182,12cyclictest9967-21apt-get09:05:122
2470599182,12cyclictest9967-21apt-get09:05:112
2470599182,11cyclictest5522-21apt-get07:40:122
2470599182,11cyclictest3533-21apt-get08:50:142
2470599181,8cyclictest13331-21rm09:58:342
24705991813,3cyclictest869-21sh11:00:362
24705991813,3cyclictest22684-21sh09:17:242
24705991811,5cyclictest908-21seq08:42:142
24705991811,5cyclictest4480-21cat07:35:242
24705991811,5cyclictest23075-21expr08:20:142
24705991811,5cyclictest19288-21cat08:10:172
24705991811,5cyclictest12146-21apt-get07:55:112
24705991810,6cyclictest11363-21cat10:45:132
2470499189,7cyclictest10347-21fw_packets10:20:171
2470499189,6cyclictest8561-21latency07:45:171
2470499189,6cyclictest25517-21grep08:25:161
2470499188,7cyclictest7840-21grep09:00:121
2470499187,9cyclictest0-21swapper/111:21:491
2470499187,3cyclictest158882sh09:36:101
2470499183,11cyclictest75-21kswapd011:05:121
2470499182,12cyclictest5144-31updatedb.mlocat07:39:051
24704991813,3cyclictest12119-21/usr/sbin/munin07:55:221
24704991811,5cyclictest21595-21cat09:40:251
24704991811,4cyclictest11859-21diskmemload10:42:051
24704991810,6cyclictest4319-21sed08:50:201
24704991810,6cyclictest15939-21cat12:00:161
24704991810,6cyclictest11922-21grep07:55:001
2470399188,6cyclictest52750irq/139-enp3s0-11:55:110
24703991814,2cyclictest23707-21df_inode10:30:150
24703991813,3cyclictest5595-21cpuspeed_turbos07:40:130
24703991813,3cyclictest3536-21sh11:50:460
24703991813,3cyclictest23560-21uptime12:05:260
24703991813,3cyclictest1086-21sh10:37:080
24703991812,4cyclictest6670-21needreboot11:05:190
24703991812,4cyclictest3777-21sh10:15:190
24703991811,5cyclictest19322-21expr11:15:130
24703991811,5cyclictest12293-21expr07:55:130
24703991810,6cyclictest5540-21date08:55:010
24703991810,6cyclictest32597-21seq08:40:210
24703991810,6cyclictest21254-21ls11:40:180
2470599171,11cyclictest1399-21taskset08:45:132
24705991710,5cyclictest17004-21smartctl08:05:162
2470499179,6cyclictest29620-21grep08:35:141
2470499178,6cyclictest278-21systemd-journal07:45:001
24704991710,5cyclictest32133-21tune2fs08:40:161
2470399177,4cyclictest14497-21wc11:35:210
24703991712,3cyclictest25681-21sh09:44:130
24703991712,3cyclictest10410-21fschecks_time09:05:140
24703991712,3cyclictest10410-21fschecks_time09:05:140
24703991711,3cyclictest30374-21netstat08:35:210
24704991611,3cyclictest14279-21/usr/sbin/munin08:00:221
24704991610,4cyclictest6460-21missed_timers08:55:201
24704991610,4cyclictest1624-21expr08:45:141
24703991610,4cyclictest8943-21grep09:00:230
24704991511,2cyclictest10292-21df09:05:141
24704991511,2cyclictest10292-21df09:05:131
24704991510,3cyclictest26635-21latency_hist07:15:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional