You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-29 - 17:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Fri May 29, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
246382183101,25sleep20-21swapper/207:08:342
24557217597,24sleep30-21swapper/307:07:313
246622140104,25sleep10-21swapper/107:08:541
245222136100,26sleep00-21swapper/007:07:040
2522699124119,3cyclictest28664-21lspci11:30:411
235522620,1sleep20-21swapper/212:15:122
188262570,1sleep20-21swapper/209:48:202
92582560,1sleep10-21swapper/111:17:001
74302560,2sleep00-21swapper/010:04:070
319922540,1sleep30-21swapper/310:21:393
168922540,1sleep30-21swapper/310:10:243
19912530,2sleep20-21swapper/212:22:542
125532520,2sleep10-21swapper/110:07:291
170752510,1sleep00-21swapper/011:45:410
69312490,9sleep228-21ksoftirqd/210:03:372
39322490,1sleep30-21swapper/310:49:373
21222480,1sleep00-21swapper/009:11:430
59452470,1sleep10-21swapper/112:01:221
105362460,1sleep30-21swapper/309:18:233
211502430,1sleep20-21swapper/212:36:482
187072390,2chrt0-21swapper/312:35:153
289992330,10sleep334-21ksoftirqd/309:55:243
325142320,1sleep20-21swapper/210:45:462
269072320,4sleep00-21swapper/009:54:400
53692310,2chrt0-21swapper/112:25:181
196382300,2chrt0-21swapper/210:12:522
291402290,2sleep10-21swapper/110:44:091
2522699298,9cyclictest22-21ksoftirqd/110:18:021
25226992924,3cyclictest211rcuc/110:27:531
25226992924,3cyclictest211rcuc/109:45:521
25225992922,5cyclictest25187-21packagekitd07:10:000
111642290,1sleep30-21swapper/311:18:553
2522699289,9cyclictest22-21ksoftirqd/109:40:041
2522699289,8cyclictest22-21ksoftirqd/110:24:511
2522699288,10cyclictest22-21ksoftirqd/112:13:581
2522699286,9cyclictest22-21ksoftirqd/111:11:051
2522699286,8cyclictest22-21ksoftirqd/110:55:001
2522699284,8cyclictest22-21ksoftirqd/111:04:211
2523299271,9cyclictest21385-21basename11:49:593
25232992716,9cyclictest7118-21ssh12:26:173
25232992714,11cyclictest34-21ksoftirqd/311:55:003
2522699279,4cyclictest22-21ksoftirqd/111:41:541
2522699278,9cyclictest22-21ksoftirqd/110:39:331
2522699278,9cyclictest22-21ksoftirqd/109:27:041
2522699274,10cyclictest22-21ksoftirqd/112:15:571
2522699272,4cyclictest221ksoftirqd/110:45:171
2522699271,8cyclictest22-21ksoftirqd/109:20:101
114792270,8sleep228-21ksoftirqd/211:19:132
38282260,2sleep10-21swapper/111:36:061
2523299267,17cyclictest24529-21perf11:04:593
2523299261,24cyclictest0-21swapper/307:13:153
2522699269,8cyclictest22-21ksoftirqd/111:56:351
2522699269,8cyclictest22-21ksoftirqd/111:56:341
2522699269,8cyclictest22-21ksoftirqd/109:31:481
2522699268,11cyclictest22-21ksoftirqd/110:34:541
2522699267,9cyclictest22-21ksoftirqd/111:53:121
2522699267,9cyclictest22-21ksoftirqd/109:59:111
2522699267,10cyclictest22-21ksoftirqd/112:36:001
2522699266,7cyclictest22-21ksoftirqd/110:01:101
2522699265,9cyclictest22-21ksoftirqd/111:20:011
2522699263,12cyclictest22-21ksoftirqd/112:20:371
25226992622,3cyclictest211rcuc/111:26:171
2522699260,7cyclictest22-21ksoftirqd/112:05:071
151942260,5sleep334-21ksoftirqd/309:21:163
73802250,1sleep30-21swapper/310:27:453
25232992512,11cyclictest34-21ksoftirqd/312:20:003
25232992511,11cyclictest21133-21ssh10:37:523
2522999251,23cyclictest0-21swapper/207:12:562
2522699257,9cyclictest22-21ksoftirqd/108:01:121
2522699256,9cyclictest22-21ksoftirqd/112:31:331
2522699256,9cyclictest22-21ksoftirqd/110:12:051
2522699256,9cyclictest22-21ksoftirqd/109:19:421
2522699256,8cyclictest22-21ksoftirqd/109:36:221
2522699252,7cyclictest22-21ksoftirqd/111:05:141
2522699250,24cyclictest0-21swapper/107:13:141
2522599251,5cyclictest17767-21sed12:35:000
2523299244,16cyclictest3144-21cron07:45:003
2523299241,6cyclictest10364-21df12:05:143
25232992415,6cyclictest31643-21grep09:10:123
25232992415,6cyclictest21996-21ntp_states09:50:203
2523299241,12cyclictest38272ssh11:36:063
25229992418,4cyclictest4514-21cstates07:50:092
2522699248,8cyclictest22-21ksoftirqd/111:48:131
2522699247,9cyclictest22-21ksoftirqd/109:12:191
2522699244,7cyclictest22-21ksoftirqd/110:56:271
2522699244,4cyclictest22-21ksoftirqd/107:50:001
25226992417,5cyclictest22-21ksoftirqd/108:25:001
2522699240,8cyclictest22-21ksoftirqd/107:40:211
2522599242,15cyclictest16995-21ssh10:35:120
25225992416,6cyclictest13944-21ssh11:44:290
25225992413,8cyclictest12427-21df_inode08:15:120
210192240,1sleep30-21swapper/309:25:253
207452240,2sleep00-21swapper/010:13:580
312482230,2sleep00-21swapper/011:32:480
2523299231,7cyclictest0-21swapper/311:55:133
2523299231,7cyclictest0-21swapper/311:55:133
25232992314,7cyclictest24790-21ntp_states10:40:223
2522699234,10cyclictest22-21ksoftirqd/109:50:451
25226992319,3cyclictest211rcuc/107:31:201
25225992317,4cyclictest12-21ksoftirqd/012:00:000
25225992317,4cyclictest12-21ksoftirqd/012:00:000
145082230,4sleep214526-21ssh10:32:592
142362230,2sleep00-21swapper/010:56:280
88732221,3sleep28886-21munin-run08:05:002
76122220,1sleep30-21swapper/310:04:183
2523299222,5cyclictest11649-21cstates10:55:133
2523299222,14cyclictest26093-21ssh09:30:113
2523299221,11cyclictest29570-21kworker/u8:212:10:103
2522999221,8cyclictest25314-21grep10:40:252
25229992215,5cyclictest765-21ssh11:34:452
25229992213,7cyclictest29313-21df_inode11:55:142
25229992213,7cyclictest29313-21df_inode11:55:132
25229992213,5cyclictest31079-21diskmemload09:35:112
2522699226,10cyclictest22-21ksoftirqd/108:42:291
25226992217,3cyclictest211rcuc/107:25:551
25226992213,2cyclictest141rcu_preempt08:47:561
25225992215,5cyclictest28575-21systemctl10:20:000
25225992214,6cyclictest2155-21ntpq07:40:200
25225992213,7cyclictest560-21dbus-daemon10:45:000
25225992213,5cyclictest52750irq/139-enp3s0-11:05:110
166312220,2sleep20-21swapper/209:22:432
161462220,8sleep334-21ksoftirqd/310:34:363
2523299217,12cyclictest17077-21rm11:23:053
2523299214,3cyclictest141rcu_preempt10:08:093
25232992114,5cyclictest22175-21/usr/sbin/munin10:15:203
25232992113,6cyclictest11627-21seq09:42:523
25229992113,6cyclictest31900-21entropy09:10:152
25229992113,6cyclictest28739-21ntp_states09:55:212
25229992113,6cyclictest23959-21ls08:50:242
25229992113,6cyclictest22862-21idleruntime10:15:162
25229992112,7cyclictest21764-21python308:45:192
25229992112,6cyclictest25954-21seq11:52:382
25229992112,6cyclictest20810-21grep09:25:242
25229992112,6cyclictest10221-21expr12:05:122
2522999211,14cyclictest11299-21ssh12:30:122
2522699218,10cyclictest211rcuc/108:05:171
2522699215,4cyclictest636-21runrttasks08:26:011
25226992117,3cyclictest211rcuc/108:16:561
25226992112,2cyclictest141rcu_preempt08:55:091
25226992112,2cyclictest141rcu_preempt08:55:091
2522699211,14cyclictest640-21snmpd08:10:031
2522699210,7cyclictest22-21ksoftirqd/109:00:161
2522699210,6cyclictest24624-21cron08:55:001
2522599216,10cyclictest52750irq/139-enp3s0-12:00:260
2522599215,4cyclictest141rcu_preempt12:15:240
2522599214,3cyclictest243872cat12:40:000
2522599212,13cyclictest17844-21chrt08:35:100
25225992113,6cyclictest18112-21cat12:10:220
25225992113,6cyclictest10656-21seq08:05:250
25225992113,5cyclictest15183-21latency_hist10:10:020
25225992112,6cyclictest5461-21grep12:25:190
25225992111,6cyclictest52750irq/139-enp3s0-11:25:120
2522599210,4cyclictest0-21swapper/009:24:000
235232210,2sleep2271rcuc/210:40:012
15932210,8sleep012-21ksoftirqd/010:47:170
255902200,1sleep30-21swapper/311:29:553
2523299208,3cyclictest34-21ksoftirqd/311:08:203
2523299201,7cyclictest11343-21latency_hist08:10:003
25232992014,4cyclictest32262-21cut11:10:143
25232992012,6cyclictest6356-21ssh12:01:463
25232992012,6cyclictest27384-21df09:00:133
25232992012,5cyclictest278-21systemd-journal08:45:003
25232992012,4cyclictest27639-21sort11:30:253
2522999201,7cyclictest8755-21grep09:40:232
25229992014,4cyclictest7656-21sh10:51:372
25229992014,4cyclictest30259-21processes09:05:222
25229992012,6cyclictest3451-21expr12:00:132
25229992012,6cyclictest21414-21perl09:50:152
25229992012,5cyclictest31079-21diskmemload11:12:442
2522699206,7cyclictest22-21ksoftirqd/109:07:201
2522699202,4cyclictest0-21swapper/107:36:311
2522699202,10cyclictest22-21ksoftirqd/108:31:231
2522699201,6cyclictest22-21ksoftirqd/107:20:221
25226992015,3cyclictest6720-21sh08:00:001
25226992013,3cyclictest211rcuc/107:15:201
25226992011,2cyclictest141rcu_preempt08:36:111
25226992011,2cyclictest141rcu_preempt07:53:341
2522599209,8cyclictest10216-21apt-get10:30:100
2522599201,9cyclictest0-21swapper/011:03:200
2522599201,7cyclictest15269-21ssh09:45:230
25225992014,4cyclictest32694-21munin-plugin-st07:35:000
25225992014,4cyclictest31078-21idleruntime-cro09:10:000
25225992013,5cyclictest12-21ksoftirqd/011:20:000
25225992012,6cyclictest3566-21seq11:13:010
25225992012,6cyclictest27509-21sed09:30:240
25225992012,6cyclictest2276-21sed11:35:180
25225992011,7cyclictest29277-21cat07:20:240
2522599201,16cyclictest24291-21seq08:50:430
25225992011,6cyclictest22302-21cat11:50:130
25225992010,6cyclictest52750irq/139-enp3s0-09:40:120
15902200,2sleep00-21swapper/010:23:410
2523299198,8cyclictest14619-21cron08:25:003
2523299192,12cyclictest4222-21apt-get07:50:083
2523299192,11cyclictest6999-21apt-get08:00:123
2523299192,11cyclictest29192-21apt-get09:05:133
2523299191,3cyclictest11286-21apt-get12:30:133
25232991910,6cyclictest14849-21sed09:45:193
2522999191,7cyclictest31079-21diskmemload10:08:092
25229991914,3cyclictest31309-21sh09:33:382
25229991913,4cyclictest9772-21grep11:40:242
25229991913,4cyclictest4699-21sh10:25:252
25229991912,5cyclictest17944-21rt-features12:10:202
25229991911,6cyclictest9467-21df08:05:132
25229991911,6cyclictest278-21systemd-journal07:49:592
25229991911,6cyclictest25056-21expr11:05:122
25229991911,6cyclictest17578-21cron08:35:002
25229991911,6cyclictest1749-21expr11:35:122
25229991911,6cyclictest1458-21cat07:40:122
25229991911,6cyclictest12380-21cat08:15:112
25229991910,6cyclictest598-21gdbus12:27:162
2522599194,3cyclictest141rcu_preempt09:40:000
2522599192,6cyclictest5827-21apt-get09:15:110
2522599191,9cyclictest141rcu_preempt11:22:210
2522599191,9cyclictest141rcu_preempt10:54:050
2522599191,4cyclictest141rcu_preempt09:27:390
25225991913,4cyclictest24895-21cpu08:55:120
25225991913,4cyclictest24895-21cpu08:55:120
25225991913,3cyclictest31079-21diskmemload10:27:570
25225991912,5cyclictest30352-21seq07:27:330
25225991912,4cyclictest31640-21sh09:57:400
25225991911,6cyclictest6472-21seq07:56:540
25225991911,6cyclictest27447-21seq07:15:530
25225991911,6cyclictest20177-21seq08:42:490
25225991911,5cyclictest4475-21expr07:50:090
25225991910,6cyclictest27649-21cat09:00:170
2523299189,6cyclictest27776-21cron07:20:003
2523299183,3cyclictest141rcu_preempt12:20:153
2523299183,10cyclictest20612-21chrt08:45:093
25232991811,5cyclictest11873-21wc08:15:003
25232991811,4cyclictest31079-21diskmemload09:38:233
25232991811,4cyclictest11085-21sh11:41:373
25232991810,6cyclictest399-21sed07:35:013
25232991810,6cyclictest28460-21cut07:20:153
25232991810,6cyclictest24023-21sed08:50:233
2522999189,6cyclictest19595-21cat11:25:122
2522999182,5cyclictest31079-21diskmemload11:20:172
2522999181,7cyclictest31079-21diskmemload10:20:062
25229991813,3cyclictest24865-21/usr/sbin/munin08:55:232
25229991813,3cyclictest24865-21/usr/sbin/munin08:55:232
25229991813,3cyclictest20360-21sh08:45:012
25229991813,3cyclictest0-21swapper/211:00:122
25229991811,5cyclictest6138-21expr09:15:122
25229991811,5cyclictest18025-21expr08:35:122
25229991811,5cyclictest11777-21expr10:55:142
25229991811,4cyclictest15460-21sh11:45:132
25229991810,6cyclictest29531-21sed07:20:262
2522599182,11cyclictest30184-21apt-get12:20:110
25225991812,4cyclictest31079-21diskmemload12:05:280
25225991811,5cyclictest20907-21expr08:45:090
25225991811,5cyclictest15382-21expr08:25:150
25225991810,5cyclictest17731-21cat08:35:010
2523299179,6cyclictest1-21systemd07:30:013
2523299173,7cyclictest24891-21apt-get08:55:123
2523299173,7cyclictest24891-21apt-get08:55:113
25232991712,3cyclictest14852-21/usr/sbin/munin08:25:203
25232991712,3cyclictest12074-21acpi08:15:093
25232991712,3cyclictest0-21swapper/310:51:563
2522999179,6cyclictest14672-21cut08:25:002
25229991712,3cyclictest28-21ksoftirqd/207:30:002
25229991712,3cyclictest27127-21cpuspeed_turbos09:00:122
2522999171,11cyclictest11030-21apt-get08:10:032
25229991710,5cyclictest6951-21cut08:00:012
25229991710,5cyclictest30792-21basename07:30:102
25229991710,5cyclictest15159-21cat08:25:132
2522599174,2cyclictest141rcu_preempt07:50:000
25225991710,5cyclictest7427-21fschecks_time08:00:150
25232991611,3cyclictest3766-21dump-pmu-power07:49:593
2523299161,12cyclictest17604-21awk08:35:003
2522599168,6cyclictest757-21seq07:38:330
2522599168,6cyclictest14616-21cron08:25:000
2523299159,4cyclictest6730-21latency_hist08:00:003
2523299158,5cyclictest1206-21sed07:40:013
25232991510,3cyclictest17815-21/usr/sbin/munin08:35:213
2522999155,8cyclictest27787-21cron07:20:002
25229991410,3cyclictest636-21runrttasks07:39:552
25225991410,3cyclictest636-21runrttasks08:12:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional