You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 15:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Wed May 13, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31281221898,96sleep10-21swapper/107:09:271
2962121660,3sleep20-21swapper/207:05:142
2748021650,1sleep30-21swapper/310:04:363
389321400,3sleep30-21swapper/309:46:093
310352140104,25sleep30-21swapper/307:06:163
294202139101,26sleep00-21swapper/007:05:050
20042590,1sleep20-21swapper/209:20:522
20042590,1sleep20-21swapper/209:20:522
152892590,2sleep315285-21dpkg12:20:003
193702580,2sleep20-21swapper/210:45:322
203562550,1sleep30-21swapper/310:23:013
281082540,1sleep20-21swapper/210:28:512
216772530,2sleep10-21swapper/109:35:241
186162520,3sleep331595-21cyclictest11:10:053
6842510,1sleep00-21swapper/010:55:490
6502490,1sleep10-21swapper/110:32:021
73162480,3sleep334-21ksoftirqd/312:37:333
314752480,1sleep20-21swapper/210:30:272
277502480,2sleep00-21swapper/011:16:010
225362480,1sleep30-21swapper/309:35:543
319092470,1sleep30-21swapper/309:43:293
213892440,2chrt0-21swapper/211:11:212
51312430,2chrt0-21swapper/111:24:041
155512400,1sleep10-21swapper/110:19:581
199302350,1chrt0-21swapper/010:46:060
113152330,2sleep00-21swapper/011:28:320
80482300,14sleep018342-21diskmemload11:01:280
321762300,2sleep00-21swapper/010:07:330
196692290,1chrt0-21swapper/211:35:002
3159899279,16cyclictest22-21ksoftirqd/110:58:171
134562270,2sleep10-21swapper/108:55:231
87472260,3sleep122-21ksoftirqd/111:50:211
3160099261,24cyclictest0-21swapper/307:12:043
3159899263,16cyclictest18772-21ssh09:10:121
31597992618,6cyclictest2492-21ssh10:33:520
257652260,2sleep10-21swapper/109:15:141
31600992512,7cyclictest18342-21diskmemload11:20:123
3159899250,24cyclictest0-21swapper/107:11:461
3159799250,24cyclictest0-21swapper/007:11:530
31600992417,5cyclictest4588-21sed08:35:213
31600992414,7cyclictest17358-21ntp_states09:05:183
3159999241,1cyclictest0-21swapper/207:11:442
3159899248,11cyclictest211rcuc/109:25:171
3159899243,15cyclictest6527-21ssh11:25:111
31598992417,4cyclictest31270-21ssh11:43:211
31597992418,4cyclictest12-21ksoftirqd/008:00:000
31597992414,7cyclictest12530-21df07:40:140
31599992313,7cyclictest20694-21df09:35:142
31598992312,7cyclictest52850irq/140-enp3s0-10:40:111
31598992311,7cyclictest52850irq/140-enp3s0-11:00:131
31597992313,7cyclictest3682-21df08:35:110
3159799231,10cyclictest274812sh10:04:350
176142230,2sleep10-21swapper/111:09:201
163412230,1sleep20-21swapper/209:31:282
77272220,4sleep27728-21needreboot09:25:182
3160099222,16cyclictest29741-21ssh10:30:133
3160099222,14cyclictest12027-21chrt11:05:123
3160099221,9cyclictest105072sendmail_mailqu10:15:213
31600992214,6cyclictest624-21gdbus07:20:003
31600992213,7cyclictest5507-21sed10:35:223
31600992212,7cyclictest6992-21tune2fs11:25:153
31600992212,6cyclictest9101-21grep11:50:233
3159899223,14cyclictest213371ssh12:00:131
3159899222,5cyclictest141rcu_preempt10:10:121
3159899222,5cyclictest141rcu_preempt10:10:111
3159899221,6cyclictest10528-21ls12:15:231
31598992213,6cyclictest16732-21cat12:20:181
31598992210,7cyclictest18342-21diskmemload12:35:221
31598992210,3cyclictest46602plymouth07:20:231
3159799229,8cyclictest52750irq/139-enp3s0-12:30:120
3159799229,8cyclictest52750irq/139-enp3s0-12:30:110
31597992213,7cyclictest27490-21ls11:40:180
31597992213,6cyclictest20948-21ntpq11:35:210
3160099212,15cyclictest6321-21sh12:12:483
31600992111,7cyclictest13130-21ntp_states07:40:213
31600992111,5cyclictest18342-21diskmemload10:55:163
31600992110,9cyclictest1732-21sh10:09:333
31599992113,6cyclictest8809-21open_inodes08:45:212
31599992112,6cyclictest15615-21ls09:00:242
31599992112,5cyclictest27536-21grep11:40:182
31599992110,9cyclictest2435-21fschecks_count12:10:152
3159899214,5cyclictest141rcu_preempt10:25:031
3159899212,14cyclictest13569-21ssh09:30:121
3159899211,8cyclictest226442ssh10:48:501
3159899211,8cyclictest198912/usr/sbin/munin11:35:181
31598992115,4cyclictest22-21ksoftirqd/108:40:001
31598992113,6cyclictest15637-21sed07:45:221
31598992112,6cyclictest52850irq/140-enp3s0-10:20:121
31598992112,6cyclictest21595-21apt-get10:00:111
31598992111,8cyclictest12561-21ssh09:53:081
3159799218,6cyclictest18526-21cat07:55:000
3159799218,11cyclictest27986-21missed_timers09:40:180
31597992115,3cyclictest15599-21dump-pmu-power10:20:000
31597992111,6cyclictest16435-21seq09:31:340
3159799211,14cyclictest8966-21ssh12:15:110
3160099202,13cyclictest27311-21sh11:40:173
3160099202,13cyclictest21034-21apt-get12:00:123
31600992011,6cyclictest7799-21cat07:30:123
31600992010,7cyclictest18818-21mailstats10:45:223
3159999205,4cyclictest141rcu_preempt12:30:002
31599992012,6cyclictest8242-21perl07:30:162
31599992012,6cyclictest8001-21date11:25:252
31599992012,6cyclictest3267-21cat12:35:002
31599992012,6cyclictest3267-21cat12:35:002
31599992012,5cyclictest14451-21cat11:55:132
3159999201,17cyclictest27803-21cat09:40:172
31599992011,6cyclictest14908-21smartctl07:45:142
3159999200,4cyclictest0-21swapper/210:03:232
3159899209,2cyclictest22-21ksoftirqd/107:25:151
3159899208,3cyclictest22-21ksoftirqd/112:25:121
3159899208,2cyclictest22-21ksoftirqd/112:05:141
3159899203,9cyclictest22-21ksoftirqd/111:13:071
3159899202,13cyclictest5730-21apt-get08:40:131
31598992014,4cyclictest22-21ksoftirqd/111:20:001
31598992013,5cyclictest17977-21ssh11:57:191
31598992012,6cyclictest477-21if_enp3s009:20:171
31598992012,6cyclictest477-21if_enp3s009:20:171
31598992012,6cyclictest16015-21seq09:02:531
31598992011,8cyclictest32608-21ssh09:44:121
31598992011,7cyclictest1894-21tune2fs09:45:151
3159899201,14cyclictest24232-21ssh10:50:101
3159799202,12cyclictest13209-21sh11:30:120
3159799201,5cyclictest0-21swapper/009:51:530
31597992014,4cyclictest22340-21sh11:12:180
31597992012,6cyclictest5587-21seq11:24:300
31597992012,6cyclictest32653-21sensors_temp08:25:220
31597992012,6cyclictest30422-21ssh12:06:230
31597992012,6cyclictest26731-21seq08:12:290
31597992012,6cyclictest22003-21cat08:00:250
31597992012,6cyclictest12969-21ls08:55:180
31597992011,7cyclictest560-21dbus-daemon12:05:000
31597992011,7cyclictest20031-21seq12:22:550
31597992010,7cyclictest23726-21cat08:05:180
31597992010,7cyclictest15456-21mailstats09:00:220
31597992010,7cyclictest1495-21gltestperf11:45:160
31597992010,7cyclictest1495-21gltestperf11:45:150
3159799200,4cyclictest5155-21ssh10:11:170
3159799200,4cyclictest5155-21ssh10:11:160
90902190,2sleep10-21swapper/107:30:251
3160099199,7cyclictest278-21systemd-journal09:30:003
3160099199,7cyclictest18361-21cron07:55:013
3160099197,10cyclictest28919-21ssh11:17:103
31600991913,4cyclictest2042-21sh12:33:583
31600991913,4cyclictest2042-21sh12:33:573
31600991911,6cyclictest3687-21ls07:20:133
31600991911,6cyclictest3674-21pluginstate10:10:203
31600991911,6cyclictest3674-21pluginstate10:10:203
31600991911,6cyclictest29579-21seq12:05:323
31600991911,6cyclictest20448-21fw_packets11:35:163
31600991911,6cyclictest1311-21expr08:30:123
3160099191,13cyclictest15704-21ssh12:20:043
31600991910,6cyclictest23564-21hddtemp_smartct10:25:153
3159999199,7cyclictest1695-21df_inode07:15:142
3159999196,4cyclictest28-21ksoftirqd/212:05:312
3159999191,8cyclictest6100-21sh11:25:002
31599991913,4cyclictest7705-21sh12:37:572
31599991913,4cyclictest31178-21sh11:19:262
31599991913,4cyclictest20974-21sh12:00:072
31599991913,4cyclictest10246-21sh09:50:472
31599991912,5cyclictest10064-21expr07:35:142
31599991912,4cyclictest28297-21/usr/sbin/munin10:05:202
31599991911,6cyclictest4232-21ls10:35:122
31599991911,6cyclictest3010-21ls09:45:252
31599991911,6cyclictest19070-21cut07:55:152
31599991911,6cyclictest16780-21diskstats09:05:142
31599991911,6cyclictest12479-21tune2fs11:05:142
31599991911,6cyclictest12456-21expr07:40:132
31599991911,6cyclictest11774-21cat11:05:012
31599991911,6cyclictest11163-21python308:50:232
31599991911,6cyclictest10330-21cat12:15:222
31599991911,5cyclictest32296-21missed_timers08:25:182
31599991910,7cyclictest4104-21date08:35:162
31599991910,7cyclictest17987-21users07:50:262
3159899192,12cyclictest14850-21apt-get09:55:121
31598991912,5cyclictest22-21ksoftirqd/111:50:011
31598991912,5cyclictest22-21ksoftirqd/111:50:001
31598991911,6cyclictest8273-21seq10:37:551
31598991911,6cyclictest29637-21expr12:30:151
31598991911,6cyclictest29637-21expr12:30:141
31598991911,5cyclictest9060-21cat08:45:231
3159799192,11cyclictest16023-21apt-get10:20:120
3159799191,6cyclictest2277-21ls09:45:180
31597991913,4cyclictest5624-21sh09:24:300
31597991913,4cyclictest5624-21sh09:24:290
31597991913,4cyclictest25089-21sh10:25:490
31597991913,4cyclictest25062-21sh09:38:260
31597991913,4cyclictest19648-21missed_timers09:10:190
31597991913,3cyclictest9140-21sh10:38:480
31597991912,5cyclictest5805-21ls08:40:130
31597991912,5cyclictest18342-21diskmemload09:15:090
31597991911,6cyclictest9989-21cat08:50:110
31597991911,6cyclictest29707-21seq08:20:150
31597991911,6cyclictest24913-21if_enp3s010:50:160
31597991911,6cyclictest22625-21wc12:25:120
31597991911,6cyclictest15491-21cat09:55:170
3159799191,15cyclictest7433-21grep09:25:160
3159799191,15cyclictest18342-21diskmemload12:38:060
31597991910,6cyclictest278-21systemd-journal08:34:370
31262190,4sleep03110-21ssh12:10:210
3160099182,9cyclictest941-21ssh11:45:113
3160099182,9cyclictest941-21ssh11:45:113
3160099182,6cyclictest32239-21apt-get09:20:133
3160099182,6cyclictest32239-21apt-get09:20:123
31600991813,3cyclictest52950irq/141-enp3s0-12:25:143
31600991813,3cyclictest14332-21/usr/sbin/munin11:55:253
31600991813,3cyclictest14291-21acpi09:00:113
31600991813,3cyclictest11596-21turbostat.cron11:05:003
3160099181,15cyclictest13797-21ssh10:41:413
31600991811,4cyclictest18342-21diskmemload10:54:293
31600991810,6cyclictest560-21dbus-daemon07:50:003
31600991810,6cyclictest15099-21ls09:30:243
3159999189,7cyclictest17359-21python310:20:222
3159999182,11cyclictest20784-21apt-get08:00:132
31599991813,3cyclictest29921-21sh09:18:082
31599991813,3cyclictest18816-21sh09:57:412
31599991813,3cyclictest1003-21cpuspeed_turbos11:45:122
31599991813,3cyclictest1003-21cpuspeed_turbos11:45:122
31599991812,4cyclictest3028-21sh10:58:102
31599991812,4cyclictest27199-21sh10:51:392
31599991812,4cyclictest11557-21sh10:40:172
31599991812,4cyclictest11279-21sh10:15:432
31599991811,5cyclictest8901-21sendmail_mailqu11:50:212
3159999181,12cyclictest29357-21apt-get08:20:142
31599991810,6cyclictest23626-21sed08:05:172
31599991810,6cyclictest22275-21date12:25:002
31599991810,6cyclictest20560-21smartctl11:35:182
31599991810,6cyclictest19072-21expr09:10:132
3159899189,7cyclictest30664-21ls08:20:251
3159899189,6cyclictest52850irq/140-enp3s0-11:30:111
3159899189,3cyclictest19333-21latency07:55:171
3159899187,8cyclictest29385-21ssh10:05:201
31598991812,3cyclictest2708-21head08:30:241
31598991811,5cyclictest10366-21tune2fs07:35:161
31598991810,6cyclictest3552-21cat12:10:251
31598991810,6cyclictest13017-21sed07:40:191
31598991810,5cyclictest25955-21ntpq08:10:201
31598991810,5cyclictest21304-21sed08:00:161
3159799189,7cyclictest3242-21cron07:20:010
3159799189,6cyclictest5987-21df07:25:130
3159799189,6cyclictest10873-21cat10:40:130
31597991813,3cyclictest4518-21grep07:20:210
31597991813,3cyclictest17575-21sh11:09:150
31597991813,3cyclictest15466-21open_inodes11:55:210
31597991811,5cyclictest8736-21awk07:30:220
31597991811,4cyclictest18342-21diskmemload11:54:340
31597991810,6cyclictest28738-21seq08:15:460
3160099179,6cyclictest560-21dbus-daemon08:25:003
3160099179,6cyclictest31758-21cat08:25:143
3160099179,6cyclictest25835-21sed08:10:183
3160099178,7cyclictest16307-21sed09:55:243
3160099178,6cyclictest7078-21cat08:40:243
3160099178,6cyclictest11913-21basename08:55:003
3160099173,8cyclictest25513-21ssh09:15:123
31600991712,3cyclictest14353-21proc_pri11:30:233
31600991712,3cyclictest12118-21/usr/sbin/munin08:55:193
31600991711,3cyclictest9138-21ntp_kernel_err09:50:193
31600991710,5cyclictest7937-21cat08:45:123
31600991710,5cyclictest23697-21sed08:05:173
31600991710,4cyclictest18342-21diskmemload09:11:303
3159999179,6cyclictest28596-21unixbench_multi08:15:252
3159999179,6cyclictest25812-21cut08:10:182
3159999178,7cyclictest6080-21forks08:40:152
31599991711,4cyclictest3045-21fschecks_time10:10:142
31599991711,4cyclictest3045-21fschecks_time10:10:142
3159999171,12cyclictest1218-21apt-get08:30:112
31599991710,5cyclictest12366-21expr08:55:142
3159899179,6cyclictest16435-21awk09:05:091
3159899179,5cyclictest23141-21expr08:05:121
3159899172,10cyclictest1361-21apt-get07:15:121
31598991712,3cyclictest27168-21/usr/sbin/munin08:15:171
31598991710,5cyclictest31711-21expr08:25:131
3159799179,5cyclictest16604-21expr09:05:110
3159799178,6cyclictest8177-21cat08:45:140
3159799172,10cyclictest9945-21apt-get07:35:120
31597991710,5cyclictest15049-21sed07:45:160
3160099169,5cyclictest27305-21expr08:15:133
3160099168,6cyclictest5876-21expr07:25:133
3160099162,11cyclictest19020-21grep07:55:153
3160099161,11cyclictest10881-21update-cracklib07:35:213
3159999169,5cyclictest3837-21expr07:20:142
31598991612,2cyclictest16461-21/usr/sbin/munin07:50:151
31598991611,3cyclictest9982-21/usr/sbin/munin08:50:171
3160099159,4cyclictest21290-21gltestperf08:00:163
31599991510,3cyclictest6067-21fschecks_time07:25:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional