You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 16:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu May 07, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27582217297,25sleep30-21swapper/307:08:043
27455216095,47sleep00-21swapper/007:06:250
276502140103,25sleep10-21swapper/107:08:581
274052140105,24sleep20-21swapper/207:05:472
299142750,3sleep30-21swapper/312:05:193
65732640,2sleep20-21swapper/211:25:012
227052640,1sleep20-21swapper/212:24:202
97892620,2sleep10-21swapper/109:52:541
279012620,2sleep1211rcuc/110:53:001
209402530,1sleep20-21swapper/210:00:242
164292530,1sleep00-21swapper/011:08:170
164292530,1sleep00-21swapper/011:08:160
41442500,2sleep20-21swapper/210:12:082
182542500,1sleep00-21swapper/012:20:210
194432480,1sleep10-21swapper/110:46:351
92742460,1sleep00-21swapper/012:14:310
69892460,2sleep20-21swapper/210:37:542
30682460,1sleep10-21swapper/109:48:001
93952450,1sleep30-21swapper/309:29:203
136792430,1sleep00-21swapper/007:50:140
113722430,2sleep00-21swapper/010:40:320
200972410,1sleep20-21swapper/211:58:202
305072400,1sleep00-21swapper/010:07:540
258382390,3chrt0-21swapper/110:28:091
316252370,3sleep00-21swapper/012:30:190
206252370,3chrt0-21swapper/111:58:511
28004993411,20cyclictest5641-21awk08:45:221
28004993411,20cyclictest21594-21seq09:37:521
28004993411,20cyclictest21594-21seq09:37:511
184922330,1sleep20-21swapper/208:00:192
2800499329,5cyclictest9938-21grep10:40:141
2800499328,5cyclictest10807-21expr09:30:131
2800499321,8cyclictest8093-21apt-config11:50:001
28004993210,20cyclictest2252-21seq07:25:131
2800499320,8cyclictest1675-21ssh09:23:261
2800499319,5cyclictest973-21basename08:35:171
2800499317,7cyclictest29016-21cut10:30:181
2800499317,5cyclictest19137-21acpi10:00:091
2800499317,4cyclictest9990-21grep11:50:211
2800499312,3cyclictest0-21swapper/112:30:111
2800499312,12cyclictest5163-21ssh12:35:121
28004993112,17cyclictest278-21systemd-journal11:15:001
28004993112,17cyclictest278-21systemd-journal11:15:001
28004993110,5cyclictest1436-21seq12:08:291
2800499307,6cyclictest10766-21seq11:27:351
2800499307,4cyclictest26591-21expr10:05:131
2800499303,3cyclictest0-21swapper/112:15:101
2800499302,4cyclictest7051-21apt-get10:15:031
2800499301,26cyclictest12975-21gltestperf09:55:151
28004993010,17cyclictest7681-21sh11:01:231
28004993010,17cyclictest28986-21rm12:28:521
2800499299,5cyclictest3926-21expr09:25:121
2800499298,4cyclictest31537-21timerwakeupswit07:15:221
2800499295,4cyclictest24007-21latency_hist08:15:011
2800499289,3cyclictest9338-21sh12:14:351
2800499287,5cyclictest17510-21date08:00:011
2800499286,19cyclictest0-21swapper/111:30:171
2800499285,5cyclictest4737-21expr07:30:141
2800499285,5cyclictest19624-21date08:05:011
2800499284,6cyclictest9783-21cut07:40:181
2800499284,5cyclictest2888-21df08:40:131
2800499283,4cyclictest0-21swapper/108:15:091
2800499282,11cyclictest19558-21ssh12:21:131
2800499282,10cyclictest15748-21ssh09:10:131
2800499281,8cyclictest1716-21sed11:20:261
2800499281,26cyclictest0-21swapper/107:14:591
43912270,2sleep30-21swapper/311:23:003
2800499278,16cyclictest52950irq/141-enp3s0-10:10:121
2800499276,18cyclictest0-21swapper/109:40:131
2800499274,6cyclictest32219-21wc11:20:011
2800499272,9cyclictest9088-21munin-run10:40:011
2800499272,7cyclictest11802-21ssh11:05:131
2800499272,7cyclictest11802-21ssh11:05:121
2800499271,8cyclictest7947-21grep07:35:261
2800499271,8cyclictest14483-21/usr/sbin/munin10:20:191
2800499271,8cyclictest10887-21grep09:00:011
2800499271,7cyclictest23703-21grep12:00:221
2800499271,7cyclictest0-21swapper/110:55:161
28004992713,12cyclictest12084-21ntp_states07:45:201
2800499269,3cyclictest1134-21sh11:44:551
2800499266,5cyclictest29913-21unixbench-2d08:25:241
2800499266,2cyclictest0-21swapper/109:15:201
2800499261,7cyclictest20542-21sort11:35:131
28004992610,2cyclictest636-21runrttasks09:09:551
2800499252,6cyclictest30672-21apt-get08:30:121
2800499251,7cyclictest27369-21pmu-power08:20:211
2800499251,7cyclictest13746-21sort07:50:131
2800499250,7cyclictest7843-21ps08:50:201
143862250,8sleep334-21ksoftirqd/307:50:203
313732240,2sleep00-21swapper/009:20:460
28006992411,9cyclictest5309-21df11:00:133
2800699241,19cyclictest52850irq/140-enp3s0-10:40:133
2800699240,11cyclictest289162ssh10:54:023
28005992413,8cyclictest30719-21df_inode10:55:152
2800499243,4cyclictest11123-21apt-get09:00:111
2800499241,8cyclictest20004-21expr08:05:141
163412240,2sleep30-21swapper/309:34:263
28005992320,2cyclictest0-21swapper/207:14:082
28005992313,7cyclictest26729-21cat08:20:132
2800499233,6cyclictest22-21ksoftirqd/107:20:111
2800699227,5cyclictest27418-21awk10:05:223
28006992220,1cyclictest0-21swapper/307:14:093
2800599222,17cyclictest9716-21cat07:40:182
28005992213,7cyclictest22194-21memory10:25:182
28005992211,8cyclictest13589-21df09:05:162
2800399229,11cyclictest7680-21apt-get10:15:120
28006992116,4cyclictest7441-21ssh09:50:353
28006992113,6cyclictest278-21systemd-journal08:15:013
28006992113,5cyclictest24686-21missed_timers12:25:193
28006992112,7cyclictest27800-21cut10:30:003
28006992112,6cyclictest1248-21grep07:20:253
2800599218,9cyclictest30376-1kworker/u9:209:10:122
2800599214,2cyclictest141rcu_preempt09:37:432
2800599214,2cyclictest141rcu_preempt09:37:422
28005992113,6cyclictest23911-21entropy10:50:142
2800399212,14cyclictest20441-21ssh11:35:120
2800399212,13cyclictest8037-21apt-get11:50:030
28003992113,6cyclictest29465-21sort12:05:150
28003992112,7cyclictest12566-21awk09:00:240
2800399210,10cyclictest66050lldpd12:37:490
299112200,3chrt0-21swapper/008:25:250
2800699208,10cyclictest17983-21ssh11:56:143
28006992013,5cyclictest8556-21idleruntime-cro07:40:003
28006992013,5cyclictest5311-21run-parts07:30:223
28006992013,4cyclictest52850irq/140-enp3s0-10:35:103
28006992012,6cyclictest1822-21grep08:35:243
28006992012,6cyclictest17425-21cut08:00:013
28006992012,6cyclictest1496-21sed10:10:183
28006992012,6cyclictest14771-21users09:05:283
28006992012,5cyclictest31476-21cat08:30:193
28006992012,5cyclictest278-21systemd-journal12:20:013
28006992012,5cyclictest19995-21cat08:05:153
28006992012,5cyclictest14101-21vmstat09:55:263
28006992011,7cyclictest20019-21awk11:35:013
28006992011,6cyclictest17890-21expr08:00:123
2800599209,8cyclictest5883-21cat08:45:232
2800599202,15cyclictest18454-21sed11:10:012
2800599202,15cyclictest18454-21sed11:10:002
2800599202,14cyclictest53050irq/142-enp3s0-10:05:132
28005992014,4cyclictest9788-21sh11:03:282
28005992014,4cyclictest53050irq/142-enp3s0-10:45:192
28005992013,5cyclictest4381-21cat07:30:132
28005992013,5cyclictest15379-21ls10:20:172
28005992012,6cyclictest9409-21cut11:50:172
28005992012,6cyclictest8220-21fw_packets10:15:162
28005992012,6cyclictest16201-21load07:55:182
28005992012,6cyclictest14612-21ls07:50:232
28005992012,6cyclictest13868-21ls09:55:242
28005992012,6cyclictest12095-21seq12:15:342
28005992011,7cyclictest9923-21cat10:40:152
28005992011,7cyclictest3235-21cut08:40:182
2800399202,12cyclictest18178-21apt-get11:10:040
2800399202,12cyclictest18178-21apt-get11:10:040
28003992014,4cyclictest25612-21sh09:16:520
28003992014,4cyclictest2243-21sh11:20:530
28003992014,4cyclictest12-21ksoftirqd/010:35:010
28003992012,6cyclictest26931-21python311:15:230
28003992012,6cyclictest14881-21seq09:06:420
28003992012,6cyclictest12-21ksoftirqd/009:00:000
28003992011,7cyclictest9118-21grep07:40:130
28003992011,7cyclictest304-21date08:35:010
28003992011,7cyclictest23480-21seq08:11:320
2800699199,7cyclictest15223-21cat11:55:013
2800699198,9cyclictest0-21swapper/310:30:143
2800699193,2cyclictest141rcu_preempt11:38:483
2800699191,3cyclictest15157-21diskmemload11:48:103
28006991913,4cyclictest28700-21sh09:43:033
28006991912,5cyclictest30782-21ls12:30:133
28006991912,5cyclictest15842-21cat09:10:123
28006991911,6cyclictest4359-21needreboot12:10:203
28006991911,5cyclictest52850irq/140-enp3s0-10:00:173
28006991910,7cyclictest6866-21awk12:35:263
2800599195,11cyclictest25961-21df11:15:122
2800599192,15cyclictest20704-21pmu-power08:05:222
2800599192,11cyclictest21875-21apt-get08:10:122
2800599191,6cyclictest0-21swapper/211:30:282
28005991914,3cyclictest29774-21interrupts12:05:172
28005991913,4cyclictest6459-21sh11:48:272
28005991913,4cyclictest29607-21rt-features08:25:222
28005991913,4cyclictest26109-21turbostat08:20:012
28005991913,4cyclictest22425-21sh11:35:412
28005991912,5cyclictest9790-21ls08:55:182
28005991912,5cyclictest5690-21seq09:25:402
28005991912,5cyclictest2641-21sh12:33:102
28005991912,5cyclictest19813-21ntpq11:10:192
28005991912,5cyclictest19813-21ntpq11:10:182
28005991912,5cyclictest11028-21cut09:30:142
28005991912,4cyclictest30472-21sh10:31:002
28005991911,6cyclictest7162-21expr11:25:132
28005991911,6cyclictest5816-21cat09:50:142
28005991911,5cyclictest25495-21grep12:25:262
28005991910,7cyclictest23273-21memory09:15:182
28005991910,7cyclictest1886-21awk08:35:262
2800399192,12cyclictest13630-21apt-get11:30:110
2800399191,6cyclictest21792-21ssh09:14:490
28003991914,3cyclictest1707-21timerwakeupswit08:35:240
28003991913,4cyclictest7751-21needreboot11:25:180
28003991913,4cyclictest2830-21cpu10:35:110
28003991913,4cyclictest27645-21sh11:40:140
28003991913,4cyclictest2634-21ssh10:10:400
28003991912,5cyclictest8485-21seq08:52:140
28003991912,5cyclictest3005-21awk07:25:210
28003991912,5cyclictest28773-21sed07:10:170
28003991911,6cyclictest7025-21fw_packets07:35:150
28003991911,6cyclictest6341-21seq08:47:070
28003991911,6cyclictest6184-21rt-features11:00:210
28003991911,6cyclictest4451-21cut10:59:590
28003991911,6cyclictest4160-21seq08:41:340
28003991911,6cyclictest15531-21cat11:55:120
28003991911,6cyclictest13448-21seq12:16:560
28003991911,6cyclictest12687-21seq07:45:280
28003991911,6cyclictest10881-21df_inode09:30:130
28003991910,6cyclictest16048-21cat07:55:180
28003991910,5cyclictest52750irq/139-enp3s0-09:45:120
241912190,2chrt0-21swapper/209:40:112
90302180,3sleep30-21swapper/310:15:223
29572180,1sleep20-21swapper/207:25:202
2800699189,7cyclictest4690-21apt-get08:45:113
2800699187,9cyclictest52850irq/140-enp3s0-11:17:453
2800699187,3cyclictest34-21ksoftirqd/311:12:303
2800699187,3cyclictest34-21ksoftirqd/311:12:293
2800699182,11cyclictest2554-21apt-get08:40:133
2800699182,11cyclictest17017-21ssh12:20:103
2800699181,6cyclictest29286-21ssh09:20:113
2800699181,3cyclictest22217-21apt-get12:00:113
28006991812,4cyclictest29773-21sh11:41:113
28006991811,5cyclictest6964-21expr11:25:123
28006991811,5cyclictest560-21dbus-daemon08:20:073
28006991811,5cyclictest30633-21date08:30:003
28006991811,5cyclictest2897-21date07:25:183
28006991811,5cyclictest12396-21head11:05:163
28006991811,5cyclictest12396-21head11:05:153
28006991811,5cyclictest11820-21cut07:45:163
28006991811,4cyclictest15157-21diskmemload09:49:383
28006991810,6cyclictest30658-21cat10:55:143
28006991810,6cyclictest23968-21sed09:40:013
28006991810,6cyclictest23968-21sed09:40:003
28006991810,6cyclictest16264-21apt-config10:45:013
2800599181,8cyclictest30845-21ssh11:42:142
2800599181,7cyclictest32722-21grep09:45:242
28005991813,3cyclictest1014-21sendmail_mailqu07:20:222
28005991812,4cyclictest22090-21sh12:00:042
28005991811,5cyclictest7710-21ntpq08:50:192
28005991811,5cyclictest5456-21cat12:35:142
28005991811,5cyclictest3570-21grep12:10:142
28005991811,5cyclictest32259-21cut07:20:012
28005991811,4cyclictest53050irq/142-enp3s0-09:20:092
28005991810,6cyclictest11912-21ls07:45:162
2800399188,6cyclictest52750irq/139-enp3s0-09:25:110
2800399182,11cyclictest17601-21apt-get08:00:110
2800399182,11cyclictest17352-21apt-get09:35:120
2800399182,11cyclictest17352-21apt-get09:35:120
2800399181,7cyclictest24691-21ssh09:40:150
2800399181,7cyclictest15765-21grep10:20:220
28003991813,4cyclictest12-21ksoftirqd/012:05:010
28003991813,3cyclictest3080-21sh11:45:220
28003991812,4cyclictest26620-21sh10:51:450
28003991811,5cyclictest24116-21cat12:25:140
28003991811,5cyclictest10989-21seq09:54:040
28003991810,6cyclictest4544-21df07:30:140
28003991810,6cyclictest18753-21cron10:00:010
28003991810,6cyclictest17297-21cat10:45:160
28003991810,5cyclictest20861-21mailstats08:05:240
2800699179,6cyclictest30302-21expr07:15:113
2800699177,3cyclictest52850irq/140-enp3s0-09:15:433
2800699173,2cyclictest141rcu_preempt10:20:073
28006991712,3cyclictest6802-21/usr/sbin/munin08:50:143
2800699171,11cyclictest23770-21apt-get08:15:043
2800599178,6cyclictest30921-21ls08:30:122
28005991712,3cyclictest11685-21hwlatdetect09:00:162
2800399179,6cyclictest26783-21tune2fs08:20:150
2800399178,6cyclictest52750irq/139-enp3s0-10:25:110
2800399172,11cyclictest30190-21apt-get07:15:120
28003991711,4cyclictest21319-21sh10:00:470
28003991710,5cyclictest24741-21sed08:15:150
2800699168,6cyclictest11098-21/usr/sbin/munin09:00:123
28006991610,4cyclictest9230-21expr07:40:143
28005991612,3cyclictest6709-21cut07:35:132
2800399162,12cyclictest12-21ksoftirqd/007:20:120
28006991510,3cyclictest9318-21fschecks_count08:55:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional