You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 19:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat May 30, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
841623627,0sleep20-21swapper/206:53:462
304450350,0irq/25-eth10-21swapper/306:50:073
825823322,0sleep00-21swapper/006:51:560
810523324,0sleep10-21swapper/106:50:091
85199960,0cyclictest6487-21diskmemload11:20:062
85189960,0cyclictest0-21swapper/109:55:041
85189960,0cyclictest0-21swapper/109:49:421
35160,0ktimersoftd/30-21swapper/311:06:173
339960,0migration/34569-21sh10:49:443
339960,0migration/329319-21sh10:25:173
339960,0migration/328888-21sh10:44:443
339960,0migration/321844-21sh11:03:053
339960,0migration/321203-21sh11:42:273
339960,0migration/313122-21sh08:55:103
29725060,0irq/27-eth0-tx-19086-21munin-node08:39:200
29725060,0irq/27-eth0-tx-11552-21munin-node08:09:230
29715060,0irq/26-eth0-rx-24404-21irqstats10:04:273
29715060,0irq/26-eth0-rx-13984-21ssh09:56:483
29715060,0irq/26-eth0-rx-12226-21python10:54:293
259960,0migration/211088-21needreboot12:14:252
149960,0migration/06888-21sh09:34:050
149960,0migration/022433-21sh09:22:030
149960,0migration/022240-21sh09:02:550
149960,0migration/020450-21sh10:39:310
149960,0migration/013005-21sh10:35:000
149960,0migration/011321-21sh08:09:080
85199955,0cyclictest0-21swapper/210:06:102
85199950,0cyclictest0-21swapper/212:00:092
85199950,0cyclictest0-21swapper/211:27:122
85199950,0cyclictest0-21swapper/209:14:272
85189955,0cyclictest0-21swapper/112:19:001
85189950,0cyclictest0-21swapper/111:53:281
85189950,0cyclictest0-21swapper/110:56:331
85189950,0cyclictest0-21swapper/110:33:031
85189950,0cyclictest0-21swapper/109:30:191
85169955,0cyclictest0-21swapper/011:12:180
85169955,0cyclictest0-21swapper/009:35:490
85169950,0cyclictest0-21swapper/011:05:120
8215050,0irq/24-0000:00:3700-21python11:29:272
339950,0migration/39124-21kthreadcore11:34:263
339950,0migration/38863-21kthreadcore12:14:243
339950,0migration/332634-21kthreadcore11:09:233
339950,0migration/331660-21gltestperf10:29:213
339950,0migration/330504-21fschecks_time12:09:213
339950,0migration/329834-21cpuspeed_turbos07:34:233
339950,0migration/329436-21sshd12:08:393
339950,0migration/328652-21kthreadcore09:24:253
339950,0migration/328174-21sshd07:05:403
339950,0migration/324204-21ksmtuned08:14:383
339950,0migration/323968-21irqcore07:29:243
339950,0migration/322244-21kthreadcore07:49:283
339950,0migration/321955-21runrttasks10:22:093
339950,0migration/319813-21gltestperf08:39:213
339950,0migration/316199-21if_eth109:39:233
339950,0migration/314603-21kthreadcore08:09:253
339950,0migration/313168-21gltestperf08:34:233
339950,0migration/311658-21missed_timers11:14:263
339950,0migration/310967-21taskset08:05:053
339950,0migration/310426-21kthreadcore10:34:263
29725050,0irq/27-eth0-tx-7721-21irqstats10:54:232
29725050,0irq/27-eth0-tx-3514-21snmpd12:20:392
29725050,0irq/27-eth0-tx-32186-21perl09:29:222
29725050,0irq/27-eth0-tx-16316-21munin-node07:24:200
29725050,0irq/27-eth0-tx-15408-21munin-node06:59:280
29725050,0irq/27-eth0-tx-14853-21munin-node08:59:302
29725050,0irq/27-eth0-tx-14705-21munin-node10:39:252
29725050,0irq/27-eth0-tx-12053-21munin-node11:14:302
29725050,0irq/27-eth0-tx-0-21swapper/210:24:222
29725050,0irq/27-eth0-tx-0-21swapper/210:19:582
29725050,0irq/27-eth0-tx-0-21swapper/209:55:362
29725050,0irq/27-eth0-tx-0-21swapper/209:52:362
29725050,0irq/27-eth0-tx-0-21swapper/209:24:192
29725050,0irq/27-eth0-tx-0-21swapper/208:57:002
29725050,0irq/27-eth0-tx-0-21swapper/008:51:540
29725050,0irq/27-eth0-tx-0-21swapper/008:24:190
29725050,0irq/27-eth0-tx-0-21swapper/006:56:340
29715050,0irq/26-eth0-rx-8348-21kthreadcore08:29:253
29715050,0irq/26-eth0-rx-31177-21irqcore11:49:233
29715050,0irq/26-eth0-rx-2988-21grep08:49:263
29715050,0irq/26-eth0-rx-23767-21fschecks_count09:44:203
29715050,0irq/26-eth0-rx-22223-21ksmtuned10:43:393
29715050,0irq/26-eth0-rx-2034-21kthreadcore09:29:253
29715050,0irq/26-eth0-rx-16860-21fschecks_time07:24:213
29715050,0irq/26-eth0-rx-0-21swapper/310:14:163
29715050,0irq/26-eth0-rx-0-21swapper/309:52:233
29715050,0irq/26-eth0-rx-0-21swapper/309:34:213
29715050,0irq/26-eth0-rx-0-21swapper/309:11:073
29715050,0irq/26-eth0-rx-0-21swapper/308:59:273
29715050,0irq/26-eth0-rx-0-21swapper/308:24:253
259950,0migration/29301-21kthreadcore10:14:222
259950,0migration/29279-21sh09:34:252
259950,0migration/24044-21aten_r1power_vo07:39:212
259950,0migration/231305-21munin-node10:09:192
259950,0migration/230490-21fschecks_time12:09:212
259950,0migration/225323-21kthreadcore11:04:242
259950,0migration/223214-21latency_hist09:24:082
259950,0migration/223208-21sh09:44:102
259950,0migration/222193-21df_abs12:04:202
259950,0migration/221272-21sh09:59:442
259950,0migration/220359-21kthreadcore06:59:262
259950,0migration/217437-21irqcore07:24:232
249950,0watchdog/224394-21cat09:04:262
179950,0migration/127188-21kthreadcore10:04:281
179950,0migration/122679-21needreboot07:49:291
169950,0watchdog/116231-21kthreadcore10:19:261
169950,0watchdog/10-21swapper/108:59:261
149950,0migration/07866-21kthreadcore08:29:240
149950,0migration/07417-21df_abs09:54:210
149950,0migration/06807-21sh10:54:200
149950,0migration/0627-21kthreadcore11:49:240
149950,0migration/032725-21kthreadcore11:29:220
149950,0migration/032098-21taskset08:48:210
149950,0migration/029849-21sh09:45:410
149950,0migration/029797-21taskset11:28:310
149950,0migration/029498-21unixbench_multi09:24:300
149950,0migration/028190-21kthreadcore09:04:280
149950,0migration/025180-21kthreadcore10:04:270
149950,0migration/023341-21kthreadcore08:14:250
149950,0migration/023203-21cpuspeed_turbos07:29:200
149950,0migration/019701-21timerandwakeup11:59:290
149950,0migration/019067-21kthreadcore09:59:260
149950,0migration/017037-21taskset07:48:530
149950,0migration/016671-21kthreadcore09:39:250
149950,0migration/014397-21latency_hist10:59:080
149950,0migration/012769-21sh11:56:520
149950,0migration/012016-21munin-node11:14:290
149950,0migration/01175-21kthreadcore09:09:250
149950,0migration/01143-21kthreadcore10:49:260
149950,0migration/010923-21kthreadcore10:14:230
149950,0migration/010516-21kthreadcore12:14:240
85199943,0cyclictest3556-21lldpd09:39:112
85199943,0cyclictest3556-21lldpd06:54:082
85199943,0cyclictest26694-21awk08:44:222
85199943,0cyclictest23716-21sadc07:54:082
85199943,0cyclictest23569-21awk10:44:222
85199943,0cyclictest12389-21awk08:09:222
85189944,0cyclictest0-21swapper/112:01:111
85189944,0cyclictest0-21swapper/111:19:151
85189944,0cyclictest0-21swapper/108:19:221
85189943,0cyclictest7981-21grep10:34:251
85189943,0cyclictest7653-21awk08:54:241
85189943,0cyclictest3949-21awk07:14:241
85189943,0cyclictest3556-21lldpd11:55:411
85189943,0cyclictest3556-21lldpd11:40:111
85189943,0cyclictest3556-21lldpd11:34:111
85189943,0cyclictest3556-21lldpd09:37:111
85189943,0cyclictest3556-21lldpd07:34:081
85189943,0cyclictest3514-21snmpd09:48:491
85189943,0cyclictest3514-21snmpd08:25:031
85189943,0cyclictest3241-21irqbalance08:03:171
85189943,0cyclictest32106-21awk10:09:221
85189943,0cyclictest11088-21grep07:44:221
85169944,0cyclictest0-21swapper/012:08:250
85169944,0cyclictest0-21swapper/011:34:220
85169944,0cyclictest0-21swapper/010:30:400
85169943,0cyclictest3556-21lldpd11:19:120
85169943,0cyclictest3514-21snmpd07:20:330
85169943,0cyclictest30508-21awk07:34:260
85169943,0cyclictest23863-21awk10:24:220
85169943,0cyclictest15772-21awk10:19:240
85169943,0cyclictest14547-21awk12:19:210
85169940,0cyclictest30357-21awk07:59:080
85169940,0cyclictest0-21swapper/009:16:350
339940,0migration/37920-21kthreadcore11:54:243
339940,0migration/329149-21kthreadcore08:44:243
339940,0migration/324174-21kthreadcore11:44:233
339940,0migration/320416-21sh11:19:373
339940,0migration/320006-21diskmemload11:59:383
339940,0migration/316185-21irqcore09:59:253
29725040,0irq/27-eth0-tx-4759-21ssh10:50:162
29725040,0irq/27-eth0-tx-0-21swapper/211:55:032
29725040,0irq/27-eth0-tx-0-21swapper/211:49:132
29725040,0irq/27-eth0-tx-0-21swapper/211:44:262
29725040,0irq/27-eth0-tx-0-21swapper/211:39:222
29725040,0irq/27-eth0-tx-0-21swapper/211:34:122
29725040,0irq/27-eth0-tx-0-21swapper/211:09:122
29725040,0irq/27-eth0-tx-0-21swapper/209:09:092
29725040,0irq/27-eth0-tx-0-21swapper/008:34:100
29725040,0irq/27-eth0-tx-0-21swapper/007:49:090
29715040,0irq/26-eth0-rx-0-21swapper/312:20:113
29715040,0irq/26-eth0-rx-0-21swapper/311:29:233
29715040,0irq/26-eth0-rx-0-21swapper/310:10:563
29715040,0irq/26-eth0-rx-0-21swapper/309:19:233
29715040,0irq/26-eth0-rx-0-21swapper/309:15:243
29715040,0irq/26-eth0-rx-0-21swapper/309:04:313
29715040,0irq/26-eth0-rx-0-21swapper/307:54:253
259940,0migration/2909-21kthreadcore07:59:252
259940,0migration/26142-21kthreadcore07:14:262
259940,0migration/232229-21irqrtprio10:29:232
259940,0migration/228869-21sh07:09:202
259940,0migration/226266-21kthreadcore07:29:252
259940,0migration/224407-21kthreadcore07:04:262
259940,0migration/223355-21kthreadcore08:14:252
259940,0migration/22078-21kthreadcore08:24:242
259940,0migration/214735-21cpuspeed_turbos10:59:192
259940,0migration/21333-21kthreadcore08:49:252
259940,0migration/211743-21kthreadcore10:34:262
249940,0watchdog/215253-21grep07:44:272
249940,0watchdog/20-21swapper/208:34:262
179940,0migration/17530-21kthreadcore08:29:241
179940,0migration/130790-21aten_r1power_fr10:49:201
179940,0migration/129639-21taskset10:26:131
179940,0migration/126450-21kthreadcore07:54:251
179940,0migration/125144-21kthreadcore11:24:241
179940,0migration/123612-21munin-node10:44:221
179940,0migration/121394-21munin-node06:59:331
179940,0migration/120986-21taskset11:00:371
179940,0migration/120527-21kthreadcore08:39:241
179940,0migration/117757-21kthreadcore09:59:261
179940,0migration/115775-21kthreadcore08:09:251
179940,0migration/11004-21irqcore08:49:241
169940,0watchdog/16559-21runrttasks11:30:461
169940,0watchdog/15324-21ssh09:10:021
169940,0watchdog/130659-21ssh11:09:181
169940,0watchdog/121676-21ssh09:40:261
169940,0watchdog/121218-21ssh12:23:421
169940,0watchdog/118060-21kthreadcore09:19:261
169940,0watchdog/116649-21cut08:34:261
169940,0watchdog/111682-21cut09:14:261
169940,0watchdog/110668-21timerandwakeup08:04:311
169940,0watchdog/10-21swapper/112:09:181
169940,0watchdog/10-21swapper/110:14:221
149940,0migration/05112-21sh09:49:350
149940,0migration/031555-21kthreadcore07:09:260
149940,0migration/030595-21timerandwakeup08:19:310
149940,0migration/025633-21kthreadcore11:44:230
149940,0migration/01386-21kthreadcore10:09:240
85199932,0cyclictest3556-21lldpd08:19:382
85199932,0cyclictest30454-21if_err_eth1.207:34:262
85199932,0cyclictest20042-21awk08:39:222
85199930,0cyclictest24644-21perl07:54:222
85199930,0cyclictest0-21swapper/208:29:252
85189932,0cyclictest3556-21lldpd11:44:121
85189932,0cyclictest23741-21grep09:24:201
85189932,0cyclictest2154-21grep07:09:321
85189932,0cyclictest15131-21forks10:39:221
85169933,0cyclictest26485-21kworker/0:212:09:130
85169933,0cyclictest26485-21kworker/0:210:44:110
85169933,0cyclictest26485-21kworker/0:208:54:280
85169933,0cyclictest26485-21kworker/0:207:59:120
85169933,0cyclictest26485-21kworker/0:207:39:180
85169933,0cyclictest26485-21kworker/0:207:14:160
85169933,0cyclictest26485-21kworker/0:207:04:220
85169932,0cyclictest14055-21munin-node11:39:170
339930,0migration/310022-21taskset07:40:113
29715030,0irq/26-eth0-rx-0-21swapper/311:24:293
29715030,0irq/26-eth0-rx-0-21swapper/308:19:233
29715030,0irq/26-eth0-rx-0-21swapper/307:48:223
29715030,0irq/26-eth0-rx-0-21swapper/307:19:143
29715030,0irq/26-eth0-rx-0-21swapper/307:09:203
29715030,0irq/26-eth0-rx-0-21swapper/306:59:203
29715030,0irq/26-eth0-rx-0-21swapper/306:56:403
259930,0migration/25477-21fschecks_time08:04:212
249930,0watchdog/26559-21runrttasks07:19:152
19130,0ktimersoftd/10-21swapper/107:23:051
179930,0migration/122448-21diskmemload11:04:091
179930,0migration/122134-21aten_r1power_fr07:04:201
169930,0watchdog/124011-21timerwakeupswit08:14:311
169930,0watchdog/10-21swapper/112:04:221
169930,0watchdog/10-21swapper/111:14:181
169930,0watchdog/10-21swapper/109:04:341
169930,0watchdog/10-21swapper/108:44:221
169930,0watchdog/10-21swapper/107:39:231
169930,0watchdog/10-21swapper/107:29:111
169930,0watchdog/10-21swapper/107:24:191
169930,0watchdog/10-21swapper/106:54:271
339920,0migration/33357-21diskstats07:14:213
339920,0migration/331574-21iostat_ios07:59:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional