You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 04:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Dec 03, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
832426564,0sleep20-21swapper/218:38:352
805425958,0sleep30-21swapper/318:35:283
829424233,0sleep10-21swapper/118:38:141
819823822,0sleep00-21swapper/018:37:050
249970,0watchdog/2213672chrt20:45:242
249970,0watchdog/20-21swapper/221:56:032
84519960,0cyclictest0-21swapper/223:44:502
84519960,0cyclictest0-21swapper/220:59:372
84489965,0cyclictest14814-21ps23:24:300
84489960,0cyclictest0-21swapper/023:56:250
84489960,0cyclictest0-21swapper/023:29:560
35160,0ktimersoftd/30-21swapper/320:48:513
339960,0migration/37514-21fschecks_time20:39:273
339960,0migration/33875-21sh23:37:093
339960,0migration/331261-21kthreadcore19:19:323
339960,0migration/330659-21sh21:33:063
339960,0migration/328752-21sh22:30:193
339960,0migration/325117-21munin-node20:24:363
339960,0migration/324087-21unixbench-2d19:59:363
339960,0migration/314535-21sh21:02:443
28415060,0irq/27-eth0-tx-28876-21ssh21:49:362
28415060,0irq/27-eth0-tx-15317-21munin-node18:44:280
28415060,0irq/27-eth0-tx-0-21swapper/223:40:262
28415060,0irq/27-eth0-tx-0-21swapper/223:12:312
28415060,0irq/27-eth0-tx-0-21swapper/222:59:462
28415060,0irq/27-eth0-tx-0-21swapper/222:41:442
28415060,0irq/27-eth0-tx-0-21swapper/222:30:472
28415060,0irq/27-eth0-tx-0-21swapper/222:16:562
28415060,0irq/27-eth0-tx-0-21swapper/222:04:452
28415060,0irq/27-eth0-tx-0-21swapper/221:59:182
28415060,0irq/27-eth0-tx-0-21swapper/221:46:532
28415060,0irq/27-eth0-tx-0-21swapper/221:30:512
28415060,0irq/27-eth0-tx-0-21swapper/221:20:352
28415060,0irq/27-eth0-tx-0-21swapper/221:11:242
28415060,0irq/27-eth0-tx-0-21swapper/220:52:332
28415060,0irq/27-eth0-tx-0-21swapper/020:29:310
28405060,0irq/26-eth0-rx-27630-21yum23:29:383
28405060,0irq/26-eth0-rx-15009-21perl22:24:273
28405060,0irq/26-eth0-rx-12520-21users22:19:373
28405060,0irq/26-eth0-rx-0-21swapper/321:54:453
169960,0watchdog/16851-21smartctl22:39:271
169960,0watchdog/1222162chrt21:26:591
149960,0migration/021739-21sh20:46:240
149960,0migration/016059-21kthreadcore22:24:310
84519955,0cyclictest0-21swapper/223:53:402
84519955,0cyclictest0-21swapper/222:49:302
84519950,0cyclictest0-21swapper/220:39:542
84509955,0cyclictest0-21swapper/123:02:241
84509955,0cyclictest0-21swapper/122:46:001
84509955,0cyclictest0-21swapper/119:19:281
84509954,0cyclictest3212-21sh23:57:351
84509954,0cyclictest29535-21awk18:54:291
84489955,0cyclictest0-21swapper/022:44:220
84489955,0cyclictest0-21swapper/022:04:100
84489950,0cyclictest17231-21munin-node19:34:260
84489950,0cyclictest0-21swapper/023:47:360
84489950,0cyclictest0-21swapper/022:52:150
84489950,0cyclictest0-21swapper/021:46:210
339950,0migration/38652-21switchtime18:59:363
339950,0migration/34450-21taskset19:47:223
339950,0migration/33644-21kthreadcore20:34:313
339950,0migration/332569-21munin-node21:14:293
339950,0migration/332006-21taskset20:31:173
339950,0migration/331273-21kthreadcore18:54:313
339950,0migration/331178-21latency_hist20:54:133
339950,0migration/330545-21munin-node23:14:273
339950,0migration/330474-21taskset21:11:593
339950,0migration/325032-21kthreadcore19:14:293
339950,0migration/322026-21timerandwakeup19:09:373
339950,0migration/316590-21kthreadcore19:54:323
339950,0migration/316572-21irqrtprio18:44:303
339950,0migration/31637-21kthreadcore20:09:303
339950,0migration/314682-21irqrtprio23:24:303
339950,0migration/314276-21taskset21:42:443
28415050,0irq/27-eth0-tx-15399-21munin-node21:04:342
28415050,0irq/27-eth0-tx-0-21swapper/223:54:292
28415050,0irq/27-eth0-tx-0-21swapper/223:32:272
28415050,0irq/27-eth0-tx-0-21swapper/223:26:372
28415050,0irq/27-eth0-tx-0-21swapper/223:14:592
28415050,0irq/27-eth0-tx-0-21swapper/223:05:442
28415050,0irq/27-eth0-tx-0-21swapper/222:54:222
28415050,0irq/27-eth0-tx-0-21swapper/222:34:432
28415050,0irq/27-eth0-tx-0-21swapper/222:26:512
28415050,0irq/27-eth0-tx-0-21swapper/222:19:222
28415050,0irq/27-eth0-tx-0-21swapper/222:09:392
28415050,0irq/27-eth0-tx-0-21swapper/221:38:092
28415050,0irq/27-eth0-tx-0-21swapper/221:25:172
28415050,0irq/27-eth0-tx-0-21swapper/221:15:052
28415050,0irq/27-eth0-tx-0-21swapper/220:56:412
28415050,0irq/27-eth0-tx-0-21swapper/200:04:332
28415050,0irq/27-eth0-tx-0-21swapper/200:02:172
28415050,0irq/27-eth0-tx-0-21swapper/019:28:330
28405050,0irq/26-eth0-rx-9848-21irqrtprio18:39:303
28405050,0irq/26-eth0-rx-5573-21ssh22:38:433
28405050,0irq/26-eth0-rx-23484-21munin-node20:49:273
28405050,0irq/26-eth0-rx-23096-21sed22:49:283
28405050,0irq/26-eth0-rx-20631-21open_files21:44:353
28405050,0irq/26-eth0-rx-17946-21sed00:04:373
28405050,0irq/26-eth0-rx-12893-21aten_r1power_po23:44:253
28405050,0irq/26-eth0-rx-0-21swapper/323:59:303
28405050,0irq/26-eth0-rx-0-21swapper/323:54:193
28405050,0irq/26-eth0-rx-0-21swapper/323:49:493
28405050,0irq/26-eth0-rx-0-21swapper/323:39:323
28405050,0irq/26-eth0-rx-0-21swapper/323:19:213
28405050,0irq/26-eth0-rx-0-21swapper/323:04:313
28405050,0irq/26-eth0-rx-0-21swapper/322:54:263
28405050,0irq/26-eth0-rx-0-21swapper/322:39:313
28405050,0irq/26-eth0-rx-0-21swapper/322:14:263
28405050,0irq/26-eth0-rx-0-21swapper/321:37:203
28405050,0irq/26-eth0-rx-0-21swapper/321:24:333
28405050,0irq/26-eth0-rx-0-21swapper/321:05:113
259950,0migration/231571-21switchtime20:29:372
19150,0ktimersoftd/10-21swapper/120:44:081
179950,0migration/15575-21latency_hist22:59:131
179950,0migration/15419-21sh22:38:151
179950,0migration/13087-21kthreadcore20:34:301
179950,0migration/128107-21rt-features22:29:361
179950,0migration/122180-21kthreadcore23:49:301
169950,0watchdog/18205-21kthreadcore19:49:321
169950,0watchdog/11464-21cut20:54:311
169950,0watchdog/114140-21kthreadcore19:54:321
149950,0migration/09424-21kthreadcore20:39:310
149950,0migration/05711-21taskset21:57:150
149950,0migration/04313-21taskset19:45:430
149950,0migration/032321-21hwlatdetect21:14:270
149950,0migration/031835-21kthreadcore22:54:290
149950,0migration/03028-21kthreadcore21:34:300
149950,0migration/030235-21kthreadcore18:54:310
149950,0migration/029778-21aten_r1power_cu23:14:240
149950,0migration/029572-21sh21:30:010
149950,0migration/025941-21kthreadcore21:09:320
149950,0migration/024365-21kthreadcore22:29:320
149950,0migration/022374-21fschecks_time23:09:270
149950,0migration/022352-21kthreadcore19:59:310
149950,0migration/02169-21kthreadcore22:34:310
149950,0migration/021502-21taskset23:09:120
149950,0migration/020046-21kthreadcore21:04:310
149950,0migration/015332-21kthreadcore20:19:320
149950,0migration/01524-21kthreadcore20:09:300
149950,0migration/012087-21sh23:20:590
149950,0migration/01185-21irqcore20:34:290
149950,0migration/011567-21sh23:41:200
149950,0migration/010446-21sh00:00:180
84519943,0cyclictest6445-21snmpd21:39:152
84519943,0cyclictest6445-21snmpd19:54:432
84519943,0cyclictest6445-21snmpd19:09:422
84519943,0cyclictest3425-21lldpd20:36:162
84519943,0cyclictest32369-21awk20:09:282
84519943,0cyclictest31387-21awk19:44:302
84519943,0cyclictest22908-21awk18:49:312
84519943,0cyclictest19994-21awk20:24:272
84519943,0cyclictest19053-21awk19:59:282
84519943,0cyclictest14974-21awk22:44:272
84509944,0cyclictest0-21swapper/121:45:281
84509943,0cyclictest7897-21awk21:39:281
84509943,0cyclictest3425-21lldpd23:19:511
84509943,0cyclictest3425-21lldpd20:09:461
84509943,0cyclictest3425-21lldpd19:14:161
84509943,0cyclictest24416-21awk21:09:301
84509943,0cyclictest22998-21awk22:49:271
84509943,0cyclictest22662-21awk23:09:281
84509943,0cyclictest19179-21awk19:59:291
84509943,0cyclictest15648-21awk22:04:311
84509943,0cyclictest11397-21awk19:29:291
84509940,0cyclictest0-21swapper/121:15:021
84489944,0cyclictest0-21swapper/023:38:460
84489944,0cyclictest0-21swapper/022:40:260
84489944,0cyclictest0-21swapper/021:53:170
84489944,0cyclictest0-21swapper/021:27:070
84489944,0cyclictest0-21swapper/021:24:070
84489944,0cyclictest0-21swapper/020:58:260
84489943,0cyclictest6811-21awk20:14:310
84489943,0cyclictest3425-21lldpd19:42:160
84489943,0cyclictest3425-21lldpd19:19:160
84489943,0cyclictest24036-21awk20:49:300
84489943,0cyclictest22860-21awk18:49:310
84489943,0cyclictest17222-21awk19:09:300
84489940,0cyclictest0-21swapper/019:00:580
339940,0migration/39130-21kthreadcore21:19:313
339940,0migration/38568-21kthreadcore21:59:293
339940,0migration/36532-21hddtemp_smartct22:59:293
339940,0migration/329064-21sh22:10:083
339940,0migration/315946-21kthreadcore20:19:323
339940,0migration/314032-21runrttasks22:44:163
339940,0migration/313062-21kthreadcore19:04:323
339940,0migration/311016-21runrttasks19:50:353
28415040,0irq/27-eth0-tx-0-21swapper/019:54:400
28405040,0irq/26-eth0-rx-0-21swapper/323:09:183
28405040,0irq/26-eth0-rx-0-21swapper/322:04:293
28405040,0irq/26-eth0-rx-0-21swapper/321:50:183
28405040,0irq/26-eth0-rx-0-21swapper/320:14:323
259940,0migration/29827-21kthreadcore20:14:332
259940,0migration/24720-21kthreadcore18:59:302
259940,0migration/231986-21kthreadcore19:19:322
259940,0migration/222161-21kthreadcore19:34:312
249940,0watchdog/28203-21kthreadcore19:49:322
249940,0watchdog/23042-21yum23:34:392
249940,0watchdog/227147-21kthreadcore20:04:322
249940,0watchdog/226756-21kthreadcore19:39:322
249940,0watchdog/215927-21fschecks_time18:44:282
249940,0watchdog/214981-21kthreadcore19:29:322
249940,0watchdog/211685-21ssh23:19:472
249940,0watchdog/20-21swapper/220:19:322
249940,0watchdog/20-21swapper/219:04:322
249940,0watchdog/20-21swapper/218:39:282
179940,0migration/19315-21kthreadcore21:59:291
179940,0migration/13847-21latency_hist19:24:131
179940,0migration/132614-21kthreadcore21:54:301
179940,0migration/127983-21kthreadcore20:29:311
179940,0migration/127616-21kthreadcore22:09:311
179940,0migration/125845-21kthreadcore19:39:311
179940,0migration/125526-21taskset20:29:071
179940,0migration/122305-21df_abs18:49:281
179940,0migration/121398-21kthreadcore19:34:311
179940,0migration/116759-21kthreadcore18:44:311
179940,0migration/115959-21kthreadcore22:24:311
179940,0migration/115720-21cpuspeed_turbos21:04:251
179940,0migration/115313-21kthreadcore23:44:301
179940,0migration/115152-21kthreadcore20:19:321
179940,0migration/111976-21sh22:19:341
179940,0migration/110986-21kthreadcore20:59:311
169940,0watchdog/19424-21smartctl18:39:281
169940,0watchdog/16583-21diskmemload23:30:551
169940,0watchdog/14429-21ssh23:17:111
169940,0watchdog/13622-21ksmtuned22:16:391
169940,0watchdog/131824-21kthreadcore19:44:321
169940,0watchdog/127146-21kthreadcore20:04:321
169940,0watchdog/121216-21ssh20:45:001
169940,0watchdog/119690-21kthreadcore19:09:321
169940,0watchdog/114741-21cut00:04:311
169940,0watchdog/111210-21ssh00:02:391
169940,0watchdog/10-21swapper/123:35:111
169940,0watchdog/10-21swapper/123:24:311
169940,0watchdog/10-21swapper/123:04:311
169940,0watchdog/10-21swapper/121:51:351
169940,0watchdog/10-21swapper/121:35:591
169940,0watchdog/10-21swapper/121:29:471
169940,0watchdog/10-21swapper/121:21:271
169940,0watchdog/10-21swapper/120:49:201
169940,0watchdog/10-21swapper/118:59:321
149940,0migration/06449-21kthreadcore19:49:310
149940,0migration/029452-21kthreadcore20:04:320
149940,0migration/027673-21taskset23:52:220
149940,0migration/013554-21kthreadcore19:04:320
149940,0migration/012134-21sh23:00:150
84509933,0cyclictest16184-21kworker/1:220:14:151
84509933,0cyclictest16184-21kworker/1:219:04:151
84489933,0cyclictest6445-21snmpd22:04:150
84489932,0cyclictest31617-21awk22:14:270
84489932,0cyclictest23873-21awk19:14:280
84489932,0cyclictest23688-21awk22:09:290
84489932,0cyclictest18387-21sh00:05:270
84489931,0cyclictest0-21swapper/022:19:310
84489930,0cyclictest19142-21munin-node20:24:350
339930,0migration/313674-21kthreadcore19:29:313
28415030,0irq/27-eth0-tx-0-21swapper/019:29:260
28415030,0irq/27-eth0-tx-0-21swapper/018:39:250
28405030,0irq/26-eth0-rx-0-21swapper/320:04:293
28405030,0irq/26-eth0-rx-0-21swapper/319:34:143
28405030,0irq/26-eth0-rx-0-21swapper/319:25:573
28405030,0irq/26-eth0-rx-0-21swapper/318:49:503
249930,0watchdog/20-21swapper/219:25:122
249930,0watchdog/20-21swapper/219:14:242
249930,0watchdog/20-21swapper/218:54:322
169930,0watchdog/10-21swapper/123:39:231
149930,0migration/014047-21runrttasks21:01:300
84489921,0cyclictest7266-21tr21:39:250
28405020,0irq/26-eth0-rx-0-21swapper/319:39:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional