You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 15:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat May 09, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3016529786,6sleep00-21swapper/007:05:460
3035329265,9sleep30-21swapper/307:08:143
3026828972,13sleep10-21swapper/107:07:071
3034228269,9sleep20-21swapper/207:08:052
30584993919,20cyclictest0-21swapper/309:50:133
3058199290,29cyclictest0-21swapper/211:19:402
3058199250,23cyclictest0-21swapper/211:31:032
3058199240,22cyclictest0-21swapper/209:40:032
30584992322,1cyclictest0-21swapper/311:24:013
30577992320,2cyclictest31521-21expr07:10:161
3058199220,22cyclictest0-21swapper/211:57:342
3058199220,22cyclictest0-21swapper/210:05:372
3057599224,10cyclictest25445-21perf11:45:000
3057599221,3cyclictest141rcu_preempt10:01:500
30584992120,1cyclictest0-21swapper/312:09:233
3058199210,21cyclictest0-21swapper/210:10:172
3058199210,19cyclictest0-21swapper/210:57:402
3058199210,18cyclictest0-21swapper/209:38:442
3057599212,1cyclictest141rcu_preempt12:31:130
3058199200,20cyclictest0-21swapper/211:43:182
3058199200,20cyclictest0-21swapper/211:12:182
3058199200,20cyclictest0-21swapper/209:29:582
3058199200,18cyclictest0-21swapper/209:17:352
3058199200,18cyclictest0-21swapper/209:17:342
30577992018,1cyclictest18365-21apt-get09:45:121
30577992015,4cyclictest25-21ksoftirqd/112:19:041
30577992015,4cyclictest25-21ksoftirqd/112:19:031
3058199190,19cyclictest0-21swapper/211:46:052
3057799190,2cyclictest0-21swapper/111:28:541
3057799190,1cyclictest0-21swapper/111:47:051
3058499180,1cyclictest0-21swapper/311:29:173
3058199180,17cyclictest0-21swapper/211:24:072
3057799185,5cyclictest0-21swapper/109:00:311
30577991814,1cyclictest25-21ksoftirqd/109:55:501
3057799180,3cyclictest0-21swapper/112:33:071
3057799180,17cyclictest0-21swapper/111:33:331
3058499171,15cyclictest8295-21fw_packets11:15:163
3058199170,16cyclictest0-21swapper/209:30:492
3058199170,15cyclictest0-21swapper/210:54:382
3057799172,14cyclictest26793-21cat09:30:201
30577991717,0cyclictest0-21swapper/108:32:281
30577991714,2cyclictest13227-21idleruntime-cro10:40:011
30577991714,2cyclictest13227-21idleruntime-cro10:40:011
30577991714,2cyclictest0-21swapper/109:11:471
30577991713,1cyclictest25-21ksoftirqd/109:22:321
3057799170,17cyclictest0-21swapper/110:43:271
3057799170,17cyclictest0-21swapper/110:43:261
30584991616,0cyclictest0-21swapper/307:30:153
3058199160,15cyclictest0-21swapper/207:50:192
30577991612,1cyclictest25-21ksoftirqd/110:16:211
30577991610,5cyclictest12638-21apache211:57:541
3057599163,8cyclictest20949-21apache_volume10:25:150
3058499159,5cyclictest23797-21diskmemload09:28:513
3058499152,2cyclictest141rcu_preempt10:47:013
3058499151,8cyclictest12449-21/usr/sbin/munin10:20:123
30584991514,1cyclictest0-21swapper/311:51:273
30581991513,1cyclictest0-21swapper/207:20:112
30581991513,1cyclictest0-21swapper/207:20:102
3058199151,1cyclictest141rcu_preempt11:50:182
3057799151,0cyclictest141rcu_preempt11:40:101
3057799150,15cyclictest0-21swapper/110:57:341
3057799150,15cyclictest0-21swapper/109:38:521
3057799150,13cyclictest23797-21diskmemload10:02:251
30575991515,0cyclictest0-21swapper/009:25:570
30575991514,1cyclictest0-21swapper/011:19:410
30584991413,1cyclictest39-21ksoftirqd/312:25:173
30584991411,1cyclictest0-21swapper/311:32:433
30584991410,3cyclictest28777-21sh10:10:193
3058199143,7cyclictest141rcu_preempt10:45:232
30581991412,1cyclictest12929-21ssh12:35:142
30581991411,2cyclictest9019-21chrt12:32:292
30581991411,2cyclictest2206-21tr07:15:222
30581991410,2cyclictest11664-21grep12:15:142
30581991410,2cyclictest11664-21grep12:15:132
3058199140,14cyclictest0-21swapper/212:13:382
3057799142,8cyclictest25145-21/usr/sbin/munin11:05:261
30577991412,2cyclictest221irq_work/109:19:511
30577991412,2cyclictest221irq_work/109:19:501
3057599144,5cyclictest31057-21perf11:10:000
30575991414,0cyclictest0-21swapper/009:13:030
30575991413,1cyclictest0-21swapper/010:35:560
30575991413,1cyclictest0-21swapper/010:35:560
30575991411,2cyclictest19066-21grep09:45:150
30575991410,3cyclictest29819-21cat10:10:270
3057599140,2cyclictest0-21swapper/012:15:370
3057599140,2cyclictest0-21swapper/012:15:360
3057599140,13cyclictest0-21swapper/012:22:070
3058499139,3cyclictest18548-21cat11:40:173
30584991310,2cyclictest9424-21cron09:40:013
30584991310,2cyclictest3104-21turbostat07:20:003
30584991310,2cyclictest25917-21taskset09:30:143
30584991310,2cyclictest20071-21sed09:00:163
30584991310,2cyclictest19594-21sed12:00:173
30584991310,2cyclictest19326-21taskset10:05:073
30584991310,2cyclictest1707-21sed08:20:263
30584991310,2cyclictest13484-21sed12:15:253
30584991310,2cyclictest13484-21sed12:15:243
30584991310,2cyclictest12941-21basename12:35:133
3058499130,12cyclictest1339-21head09:15:183
3058499130,12cyclictest1339-21head09:15:183
3058199139,3cyclictest22966-21tr08:00:142
3058199139,3cyclictest22620-21cron11:05:002
3058199135,6cyclictest537-21snmpd09:47:402
30581991312,1cyclictest0-21swapper/211:29:552
30581991310,2cyclictest7882-21cut08:35:192
30581991310,2cyclictest6516-21grep10:35:212
30581991310,2cyclictest6516-21grep10:35:202
30581991310,2cyclictest5950-21tail10:15:262
30581991310,2cyclictest24641-21tr09:10:152
30581991310,2cyclictest11038-21grep11:35:262
3058199130,13cyclictest0-21swapper/212:29:202
3058199130,12cyclictest28678-21tail09:50:262
3057799136,4cyclictest19522-21fschecks_time12:00:171
3057799131,0cyclictest141rcu_preempt11:22:181
30577991310,2cyclictest9388-21wc07:30:221
30577991310,2cyclictest27404-21ssh10:10:001
30577991310,2cyclictest22474-21cut09:05:171
30577991310,2cyclictest21130-21grep10:25:171
30577991310,2cyclictest1520-21wc08:20:261
30577991310,2cyclictest13050-21grep10:20:141
30577991310,2cyclictest11484-21sed07:35:171
30577991310,2cyclictest10265-21tr09:40:161
3057799130,2cyclictest0-21swapper/111:35:131
3057599139,3cyclictest21917-21grep09:05:150
3057599139,3cyclictest15090-21tr10:40:210
3057599139,3cyclictest15090-21tr10:40:200
3057599138,4cyclictest12638-21apache209:43:310
3057599132,7cyclictest8134-21apt-get07:30:140
30575991312,1cyclictest603-1kworker/u9:209:53:330
30575991310,2cyclictest8882-21apache_volume09:20:130
30575991310,2cyclictest4552-21grep12:10:210
30575991310,2cyclictest30657-21rm12:25:370
3057599130,12cyclictest7200-21cat07:25:260
3058499129,2cyclictest9245-21run-parts08:40:003
3058499129,2cyclictest7820-21grep10:55:213
3058499129,2cyclictest5477-21cron07:25:013
3058499129,2cyclictest5477-21cron07:25:003
3058499129,2cyclictest5446-21cat12:30:193
3058499129,2cyclictest5213-21ssh10:35:123
3058499129,2cyclictest5213-21ssh10:35:113
3058499129,2cyclictest32455-21grep07:10:263
3058499129,2cyclictest30763-21grep10:50:143
3058499129,2cyclictest26217-21grep09:10:273
3058499129,2cyclictest26097-21wget11:45:143
3058499129,2cyclictest23934-21tr11:05:153
3058499129,2cyclictest20206-21grep12:20:113
3058499129,2cyclictest19739-21grep09:45:203
3058499129,2cyclictest15545-21grep11:00:163
3058499129,2cyclictest14650-21cat07:40:263
3058499129,2cyclictest13837-21tr10:40:133
3058499129,2cyclictest13837-21tr10:40:133
3058499129,2cyclictest10235-21grep08:40:153
3058499128,3cyclictest892-21ls11:10:253
3058499128,3cyclictest17163-21tr08:55:143
30584991211,1cyclictest0-21swapper/311:36:323
3058199129,2cyclictest6422-21cut07:25:172
3058199129,2cyclictest3538-21grep09:55:172
3058199129,2cyclictest21736-21wc07:55:262
3058199129,2cyclictest16763-21tr07:45:222
3058199129,2cyclictest15279-21grep08:50:162
3058199129,2cyclictest1199-21sed08:20:222
3058199128,3cyclictest28818-21ls12:05:252
3058199120,11cyclictest32136-21cron08:19:592
3057799129,2cyclictest3225-21tr11:50:211
3057799129,2cyclictest31839-21grep11:10:121
3057799129,2cyclictest24670-21chrt12:22:311
3057799129,2cyclictest23467-21sed08:00:171
3057799129,2cyclictest16364-21find11:00:221
3057799129,2cyclictest14576-21grep08:50:131
3057799129,2cyclictest13876-21tail07:40:171
3057799129,2cyclictest12151-21tr08:45:141
3057799129,2cyclictest10749-21tr08:40:201
3057799127,2cyclictest25-21ksoftirqd/109:26:481
3057799121,10cyclictest3034-21/usr/sbin/munin12:10:201
30577991210,1cyclictest28187-21cut08:10:171
30577991210,1cyclictest19154-21wc09:00:011
3057799120,11cyclictest20909-21cut07:55:161
3057599129,2cyclictest447-21cron07:15:000
3057599129,2cyclictest4083-21grep08:25:280
3057599129,2cyclictest30816-21grep08:15:180
3057599129,2cyclictest26620-21sh12:05:000
3057599129,2cyclictest23837-21cat10:45:230
3057599129,2cyclictest18484-21cat08:55:240
3057599129,2cyclictest11662-21wc08:45:000
3057599129,2cyclictest0-21swapper/011:31:030
3057599122,3cyclictest225-21systemd-journal11:10:240
30575991210,1cyclictest20399-21munin-run10:24:590
30575991210,1cyclictest0-21swapper/011:22:560
3058499119,1cyclictest31180-21sed10:30:253
3058499119,1cyclictest21225-21tr10:25:173
3058499119,1cyclictest11037-21ssh11:55:173
3058499118,2cyclictest9199-21tr09:20:143
3058499118,2cyclictest7845-21chrt07:29:573
3058499118,2cyclictest4746-21grep10:15:163
3058499118,2cyclictest32135-21cron08:19:593
3058499118,2cyclictest22368-21taskset07:59:473
3058499118,2cyclictest20130-21cat07:55:013
3058499118,2cyclictest12957-21grep10:00:243
3058499118,2cyclictest11693-21cat09:40:263
3058499112,3cyclictest23788-21cron09:10:003
3058199118,2cyclictest8967-21cut09:20:142
3058199118,2cyclictest32694-21sh10:31:302
3058199118,2cyclictest29537-21cron08:15:002
3058199118,2cyclictest27008-21chrt08:08:012
3058199118,2cyclictest23523-21ssh11:05:142
3058199118,2cyclictest20947-21sh10:25:152
3058199118,2cyclictest20433-21ssh12:00:262
3058199118,2cyclictest18565-21grep08:55:252
3058199118,2cyclictest14029-21cat10:20:222
3058199118,2cyclictest13621-21sh10:40:092
3058199118,2cyclictest13621-21sh10:40:092
3058199118,2cyclictest11726-21grep10:00:152
3058199110,10cyclictest25182-21sh12:22:552
3058199110,10cyclictest14492-21wc07:40:222
3057799118,2cyclictest4745-21cron10:34:591
3057799118,2cyclictest30465-21ssh10:50:121
3057799118,2cyclictest30157-21ssh12:06:131
3057799118,2cyclictest29734-21chrt10:49:491
3057799118,2cyclictest28372-21tr10:10:171
3057799118,2cyclictest18662-21cut07:50:171
3057799118,2cyclictest17594-21ssh12:37:371
3057799118,2cyclictest12988-21rm11:18:061
3057799110,9cyclictest0-21swapper/112:29:391
3057799110,3cyclictest28888-21kworker/1:209:52:231
3057599119,1cyclictest9947-21head11:35:160
3057599119,1cyclictest3186-21grep11:50:210
3057599119,1cyclictest31214-21grep10:50:160
3057599119,1cyclictest12332-21grep07:35:260
3057599118,2cyclictest7120-21ssh10:16:170
3057599118,2cyclictest6222-21ssh09:56:380
3057599118,2cyclictest4945-21timerwakeupswit07:20:250
3057599118,2cyclictest4945-21timerwakeupswit07:20:240
3057599118,2cyclictest3135-21ssh10:33:450
3057599118,2cyclictest29435-21taskset11:46:340
3057599118,2cyclictest25358-21ssh11:25:170
3057599118,2cyclictest20070-21grep09:00:170
3057599118,2cyclictest19890-21chrt07:53:260
3057599118,2cyclictest14666-21ssh11:00:040
3057599118,2cyclictest13355-21df12:35:160
3057599118,2cyclictest13099-21ntpq08:45:200
3057599111,9cyclictest0-21swapper/007:40:140
30575991110,1cyclictest0-21swapper/011:57:340
3058499108,1cyclictest27651-21ls08:10:153
3058499108,1cyclictest2610-21cut08:25:163
3058499108,1cyclictest25304-21expr08:05:163
3058499108,1cyclictest15642-21wget07:45:143
3058499108,1cyclictest12539-21tune2fs08:45:163
3058499108,1cyclictest11491-21sed07:35:173
3058499107,2cyclictest5965-21cut08:30:233
3058499107,2cyclictest2966-21rm12:10:073
3058499107,2cyclictest14830-21tr08:50:133
3058499100,9cyclictest2444-21chrt09:55:013
3058199107,2cyclictest718-21dpkg07:15:002
3058199107,2cyclictest6103-21grep08:30:232
3058199107,2cyclictest3053-21cut08:25:192
3058199107,2cyclictest12372-21sed07:35:262
3058199107,2cyclictest12319-21expr08:45:142
3058199104,5cyclictest21684-21apache_accesses09:05:132
3058199100,9cyclictest19337-21basename09:00:142
3057799108,1cyclictest7320-21grep08:35:161
3057799108,1cyclictest3733-21ls07:20:141
3057799108,1cyclictest3733-21ls07:20:141
3057799108,1cyclictest25400-21ls08:05:161
3057799107,2cyclictest7322-21wc07:25:261
3057799107,2cyclictest4295-21chrt08:27:371
3057799107,2cyclictest2636-21tail07:15:261
3057799104,4cyclictest30261-21tr08:15:151
3057599108,2cyclictest26460-21sensors08:05:250
3057599108,1cyclictest8894-21ssh10:55:490
3057599108,1cyclictest6668-21seq08:32:360
3057599108,1cyclictest32522-21sh09:15:010
3057599108,1cyclictest32522-21sh09:15:000
3057599108,1cyclictest28528-21tr08:10:200
3057599108,1cyclictest23012-21cat08:00:140
3057599108,1cyclictest21403-21ssh10:05:270
3057599107,2cyclictest348-21expr08:20:140
3057599107,2cyclictest3439-21sed09:35:260
3057599107,2cyclictest28038-21rm09:30:470
3057599107,2cyclictest27120-21apache_accesses12:05:120
3057599107,2cyclictest2526-21cat07:15:260
3057599107,2cyclictest22069-21chrt07:55:530
305849996,2cyclictest22650-21chrt08:00:103
305819997,1cyclictest9576-21cut08:40:122
305819996,2cyclictest8801-21cut07:30:172
305779997,1cyclictest16226-21grep07:45:171
305759997,1cyclictest7317-21cpu08:35:160
305759996,2cyclictest14540-21chrt08:50:120
305759987,1cyclictest15923-21cat07:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional