You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 01:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Aug 30, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1832525857,0sleep00-21swapper/006:41:100
1856623826,0sleep30-21swapper/306:43:573
1230723425,0sleep20-21swapper/206:40:242
1866023324,0sleep10-21swapper/106:45:041
339960,0migration/32942-21sh11:57:123
339960,0migration/327832-21sh11:32:203
339960,0migration/325419-21latency_hist06:50:223
339960,0migration/319794-21sh12:09:483
29725060,0irq/27-eth0-tx-27559-21munin-node07:40:440
29715060,0irq/26-eth0-rx-4236-21cat12:00:213
29715060,0irq/26-eth0-rx-1568-21idleruntime08:55:383
29715060,0irq/26-eth0-rx-10615-21aten2.4-expect08:40:343
187569960,0cyclictest0-21swapper/109:32:511
149960,0migration/06300-21sh10:40:300
149960,0migration/05589-21munin-node06:55:420
149960,0migration/032041-21sh09:34:540
149960,0migration/030329-21sh10:13:220
149960,0migration/023937-21sh09:07:310
149960,0migration/021629-21munin-node08:20:400
8215050,0irq/24-0000:00:22549-21crond08:25:212
339950,0migration/38535-21kthreadcore07:00:403
339950,0migration/33192-21kthreadcore09:15:393
339950,0migration/331518-21kthreadcore11:35:383
339950,0migration/330635-21munin-node08:30:373
339950,0migration/32976-21kthreadcore06:55:383
339950,0migration/328490-21kthreadcore06:50:393
339950,0migration/327455-21latency_hist07:40:223
339950,0migration/323465-21fschecks_time08:25:353
339950,0migration/31972-21kthreadcore09:35:383
339950,0migration/319261-21munin-node11:25:443
339950,0migration/316592-21kthreadcore10:45:383
339950,0migration/315602-21munin-node07:55:343
339950,0migration/312989-21taskset10:42:163
339950,0migration/312029-21kthreadcore10:20:383
29725050,0irq/27-eth0-tx-9696-21iostat09:20:372
29725050,0irq/27-eth0-tx-18732-21kthreadcore11:05:402
29725050,0irq/27-eth0-tx-0-21swapper/211:00:502
29725050,0irq/27-eth0-tx-0-21swapper/210:40:312
29725050,0irq/27-eth0-tx-0-21swapper/209:50:432
29725050,0irq/27-eth0-tx-0-21swapper/209:37:542
29725050,0irq/27-eth0-tx-0-21swapper/209:00:322
29725050,0irq/27-eth0-tx-0-21swapper/008:40:330
29725050,0irq/27-eth0-tx-0-21swapper/007:45:330
29725050,0irq/27-eth0-tx-0-21swapper/007:45:330
29725050,0irq/27-eth0-tx-0-21swapper/007:35:330
29715050,0irq/26-eth0-rx-9828-21if_err_eth109:00:363
29715050,0irq/26-eth0-rx-3301-21kthreadcore10:35:403
29715050,0irq/26-eth0-rx-27878-21cat09:10:383
29715050,0irq/26-eth0-rx-26081-21interrupts08:50:373
29715050,0irq/26-eth0-rx-24945-21kthreadcore11:50:383
29715050,0irq/26-eth0-rx-24671-21sed10:10:383
29715050,0irq/26-eth0-rx-22344-21sendmail_mailst09:25:423
29715050,0irq/26-eth0-rx-16142-21kthreadcore10:25:383
29715050,0irq/26-eth0-rx-1406-21kthreadcore10:55:373
29715050,0irq/26-eth0-rx-13568-21uptime07:25:433
29715050,0irq/26-eth0-rx-13070-21ntp_states10:00:423
29715050,0irq/26-eth0-rx-0-21swapper/312:10:223
29715050,0irq/26-eth0-rx-0-21swapper/311:47:253
29715050,0irq/26-eth0-rx-0-21swapper/311:43:003
29715050,0irq/26-eth0-rx-0-21swapper/311:00:373
29715050,0irq/26-eth0-rx-0-21swapper/310:35:143
29715050,0irq/26-eth0-rx-0-21swapper/310:16:213
29715050,0irq/26-eth0-rx-0-21swapper/309:50:323
29715050,0irq/26-eth0-rx-0-21swapper/309:45:333
29715050,0irq/26-eth0-rx-0-21swapper/308:48:293
27150,0ktimersoftd/20-21swapper/209:41:522
259950,0migration/27876-21unixbench_singl07:45:452
259950,0migration/27876-21unixbench_singl07:45:452
259950,0migration/231719-21sh08:52:092
259950,0migration/230114-21ntp_offset09:30:432
259950,0migration/229672-21aten_r1power_cu11:15:322
259950,0migration/229608-21fschecks_count11:35:342
259950,0migration/227309-21kthreadcore06:50:382
259950,0migration/21574-21kthreadcore09:55:382
249950,0watchdog/25710-21sh10:17:432
249950,0watchdog/22500-21kthreadcore10:35:392
249950,0watchdog/222583-21unixbench-2d09:25:442
249950,0watchdog/21397-21latency08:05:402
249950,0watchdog/210285-21kthreadcore07:00:402
19150,0ktimersoftd/10-21swapper/108:58:291
187579955,0cyclictest18138-21ps09:05:382
187579955,0cyclictest16918-21diskmemload12:11:482
187579954,0cyclictest3556-21lldpd08:28:122
187579950,0cyclictest0-21swapper/212:00:082
187579950,0cyclictest0-21swapper/211:21:592
187569955,0cyclictest0-21swapper/111:16:451
187569955,0cyclictest0-21swapper/109:11:341
187569955,0cyclictest0-21swapper/108:53:071
187569955,0cyclictest0-21swapper/108:38:371
187569950,0cyclictest0-21swapper/110:20:091
187569950,0cyclictest0-21swapper/109:35:411
187559955,0cyclictest5702-21ssh10:17:410
187559950,0cyclictest0-21swapper/009:36:420
179950,0migration/19731-21mdstat12:00:401
179950,0migration/18630-21fschecks_count09:40:351
179950,0migration/17059-21sh09:59:521
179950,0migration/15462-21cpuspeed_turbos11:20:331
179950,0migration/122965-21switchtime08:45:441
179950,0migration/12192-21kthreadcore09:15:391
179950,0migration/120011-21taskset11:49:021
179950,0migration/118337-21irqrtprio09:05:381
179950,0migration/114750-21sh10:04:101
179950,0migration/111896-21kthreadcore09:00:391
149950,0migration/09723-21kthreadcore10:00:380
149950,0migration/09576-21kthreadcore11:20:380
149950,0migration/08013-21kthreadcore08:10:390
149950,0migration/07846-21kthreadcore11:40:370
149950,0migration/07689-21aten2.4-expect07:25:310
149950,0migration/07192-21taskset07:23:500
149950,0migration/06885-21aten_r1power_cu10:20:310
149950,0migration/05555-21unixbench-2d09:55:440
149950,0migration/031357-21switchtime06:50:440
149950,0migration/031344-21taskset09:10:540
149950,0migration/030362-21kthreadcore08:05:390
149950,0migration/029565-21aten2.4-expect08:30:320
149950,0migration/028248-21taskset08:04:490
149950,0migration/027830-21kthreadcore10:30:390
149950,0migration/026434-21kthreadcore09:50:370
149950,0migration/025482-21latency_hist06:50:220
149950,0migration/024198-21kthreadcore07:10:390
149950,0migration/023633-21kthreadcore11:50:380
149950,0migration/022794-21sh09:25:570
149950,0migration/022513-21taskset10:08:370
149950,0migration/020997-21kthreadcore09:45:370
149950,0migration/020166-21kthreadcore08:45:390
149950,0migration/019266-21needreboot11:05:410
149950,0migration/018944-21kthreadcore10:25:390
149950,0migration/017688-21kthreadcore07:05:400
149950,0migration/017002-21kthreadcore12:05:390
149950,0migration/014806-21mdstat08:15:410
149950,0migration/012954-21kthreadcore07:50:380
149950,0migration/010115-21smart_sdb12:00:430
339940,0migration/36921-21kthreadcore07:45:403
339940,0migration/36921-21kthreadcore07:45:403
339940,0migration/35737-21kthreadcore08:35:383
339940,0migration/329827-21kernelversion08:05:383
339940,0migration/329072-21kthreadcore07:15:393
339940,0migration/32699-21kthreadcore09:55:383
339940,0migration/326711-21kthreadcore09:30:403
339940,0migration/322261-21diskmemload10:50:303
339940,0migration/321742-21aten_r1power_en11:10:303
339940,0migration/317780-21kthreadcore08:20:373
339940,0migration/312243-21kthreadcore08:15:393
29725040,0irq/27-eth0-tx-25921-21kthreadcore10:10:382
29725040,0irq/27-eth0-tx-0-21swapper/211:50:382
29725040,0irq/27-eth0-tx-0-21swapper/211:45:402
29725040,0irq/27-eth0-tx-0-21swapper/211:25:432
29725040,0irq/27-eth0-tx-0-21swapper/210:26:112
29725040,0irq/27-eth0-tx-0-21swapper/210:05:312
29725040,0irq/27-eth0-tx-0-21swapper/209:45:372
29725040,0irq/27-eth0-tx-0-21swapper/209:15:232
29725040,0irq/27-eth0-tx-0-21swapper/208:55:232
29725040,0irq/27-eth0-tx-0-21swapper/208:49:232
29725040,0irq/27-eth0-tx-0-21swapper/007:30:450
29715040,0irq/26-eth0-rx-2587-21cat11:15:393
29715040,0irq/26-eth0-rx-0-21swapper/311:20:243
29715040,0irq/26-eth0-rx-0-21swapper/311:05:533
29715040,0irq/26-eth0-rx-0-21swapper/310:05:483
29715040,0irq/26-eth0-rx-0-21swapper/309:42:323
29715040,0irq/26-eth0-rx-0-21swapper/309:20:253
29715040,0irq/26-eth0-rx-0-21swapper/309:05:523
29715040,0irq/26-eth0-rx-0-21swapper/307:50:343
259940,0migration/27307-21kthreadcore08:10:392
259940,0migration/25497-21gltestperf11:40:342
259940,0migration/225250-21sh09:10:322
259940,0migration/22124-21kthreadcore10:55:372
259940,0migration/216134-21kthreadcore12:05:392
259940,0migration/215598-21kthreadcore07:05:392
259940,0migration/213526-21kthreadcore08:15:402
259940,0migration/212461-21kthreadcore07:50:382
249940,0watchdog/2822-21if_eth1.306:55:362
249940,0watchdog/230340-21if_err_eth1.208:30:362
249940,0watchdog/20-21swapper/212:00:592
249940,0watchdog/20-21swapper/210:20:352
249940,0watchdog/20-21swapper/210:00:392
19140,0ktimersoftd/115841-21munin-node10:25:371
187579943,0cyclictest3556-21lldpd10:31:432
187579943,0cyclictest3556-21lldpd08:37:122
187579943,0cyclictest3556-21lldpd08:03:112
187579943,0cyclictest3556-21lldpd07:58:112
187579943,0cyclictest3556-21lldpd07:44:112
187579943,0cyclictest3556-21lldpd07:36:112
187579943,0cyclictest3556-21lldpd07:34:112
187579943,0cyclictest3556-21lldpd07:15:402
187579943,0cyclictest3556-21lldpd07:14:102
187579943,0cyclictest3514-21snmpd11:11:392
187579943,0cyclictest23164-21awk10:50:362
187579943,0cyclictest22163-21awk11:30:362
187579943,0cyclictest15634-21ps10:45:382
187579942,0cyclictest5540-21grep07:20:402
187569944,0cyclictest0-21swapper/110:34:111
187569943,0cyclictest9424-21awk07:50:351
187569943,0cyclictest3556-21lldpd11:05:131
187569943,0cyclictest3556-21lldpd07:10:401
187569943,0cyclictest3514-21snmpd11:55:391
187569943,0cyclictest3514-21snmpd11:44:391
187569943,0cyclictest3514-21snmpd11:26:091
187569943,0cyclictest3514-21snmpd10:35:381
187569943,0cyclictest3514-21snmpd10:12:381
187569943,0cyclictest3514-21snmpd09:20:371
187569943,0cyclictest3514-21snmpd08:27:371
187569943,0cyclictest3514-21snmpd08:21:371
187569943,0cyclictest3514-21snmpd08:05:371
187569943,0cyclictest3514-21snmpd08:00:371
187569943,0cyclictest3514-21snmpd07:38:371
187569943,0cyclictest3514-21snmpd07:25:371
187569943,0cyclictest3514-21snmpd07:07:361
187569943,0cyclictest14765-21awk11:05:371
187559944,0cyclictest26403-21kworker/0:108:56:310
187559944,0cyclictest0-21swapper/011:56:510
187559944,0cyclictest0-21swapper/010:59:560
187559943,0cyclictest7640-21awk07:00:380
187559943,0cyclictest3556-21lldpd11:11:130
187559943,0cyclictest3556-21lldpd09:40:430
187559943,0cyclictest3514-21snmpd08:35:370
187559943,0cyclictest3514-21snmpd07:19:510
187559943,0cyclictest3241-21irqbalance11:48:290
187559943,0cyclictest3241-21irqbalance11:34:590
187559943,0cyclictest30026-21awk11:35:360
187559943,0cyclictest14144-21awk11:25:360
179940,0migration/129861-21df_abs08:30:341
179940,0migration/12775-21taskset08:10:261
179940,0migration/124881-21kthreadcore10:50:391
179940,0migration/120367-21runrttasks07:32:091
179940,0migration/117226-21fschecks_time09:25:341
169940,0watchdog/1836-21if_eth1.306:55:361
169940,0watchdog/16559-21runrttasks11:50:271
169940,0watchdog/14469-21ssh10:57:071
169940,0watchdog/131495-21timerwakeupswit06:50:441
169940,0watchdog/130503-21ssh09:51:551
169940,0watchdog/129910-21hwlatdetect11:35:351
169940,0watchdog/127238-21ssh11:10:391
169940,0watchdog/120256-21munin-node12:10:431
169940,0watchdog/119669-21latency10:45:391
169940,0watchdog/117065-21irqcore09:45:351
169940,0watchdog/114277-21sh10:24:351
169940,0watchdog/111228-21hddtemp_smartct08:40:361
169940,0watchdog/110286-21kthreadcore07:00:401
169940,0watchdog/10-21swapper/112:09:231
169940,0watchdog/10-21swapper/111:30:231
169940,0watchdog/10-21swapper/110:41:431
169940,0watchdog/10-21swapper/110:05:391
169940,0watchdog/10-21swapper/108:15:401
169940,0watchdog/10-21swapper/107:20:401
169940,0watchdog/10-21swapper/107:15:401
149940,0migration/06843-21if_eth1.311:00:360
149940,0migration/0616-21kthreadcore10:35:390
149940,0migration/04712-21kthreadcore09:15:390
149940,0migration/025097-21kthreadcore12:10:390
149940,0migration/017637-21kthreadcore07:55:380
149940,0migration/014467-21swap09:20:440
339930,0migration/328006-21runrttasks08:02:013
29715030,0irq/26-eth0-rx-0-21swapper/308:14:383
29715030,0irq/26-eth0-rx-0-21swapper/307:40:373
29715030,0irq/26-eth0-rx-0-21swapper/307:31:153
29715030,0irq/26-eth0-rx-0-21swapper/307:20:343
29715030,0irq/26-eth0-rx-0-21swapper/307:11:403
29715030,0irq/26-eth0-rx-0-21swapper/307:05:433
187579932,0cyclictest8499-21awk07:25:352
187579932,0cyclictest20878-21cut06:45:392
187579932,0cyclictest11300-21awk08:40:362
187569932,0cyclictest3514-21snmpd07:56:371
187569932,0cyclictest3514-21snmpd06:45:361
187569932,0cyclictest27983-21fschecks_count07:40:341
187559933,0cyclictest26403-21kworker/0:111:15:260
187559933,0cyclictest26403-21kworker/0:110:50:240
187559933,0cyclictest26403-21kworker/0:110:45:220
187559933,0cyclictest26403-21kworker/0:109:00:290
187559933,0cyclictest26403-21kworker/0:108:50:250
187559932,0cyclictest22878-21hostname08:25:330
169930,0watchdog/10-21swapper/107:45:361
169930,0watchdog/10-21swapper/107:45:361
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional