You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 03:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Mon Nov 24, 2025 00:46:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30663501110irq/52-eth0-rx-0-21swapper/119:05:271
17153210894sleep80-21swapper/819:07:3710
122501020irq/42-ahci0-21swapper/219:09:454
3066350900irq/52-eth0-rx-0-21swapper/419:06:456
12250900irq/42-ahci0-21swapper/719:08:249
3066350860irq/52-eth0-rx-0-21swapper/519:07:387
12250840irq/42-ahci0-21swapper/319:07:165
1732727876sleep90-21swapper/919:09:1511
3066350730irq/52-eth0-rx-0-21swapper/1019:05:402
1711026453sleep60-21swapper/619:06:598
1719325955sleep00-21swapper/019:08:060
1706625955sleep110-21swapper/1119:06:283
17526993114cyclictest831ksoftirqd/1021:30:142
17485993117cyclictest0-21swapper/019:19:170
1752699236cyclictest831ksoftirqd/1023:37:442
1752699236cyclictest831ksoftirqd/1023:37:442
17527992221cyclictest20523-1kworker/11:2H20:20:133
1752699225cyclictest831ksoftirqd/1023:33:192
1752799210cyclictest1359-21turbostat20:19:563
1752799210cyclictest1359-21turbostat20:19:553
1752699214cyclictest831ksoftirqd/1020:55:352
17527991917cyclictest0-21swapper/1119:34:563
1748599190cyclictest0-21swapper/021:11:540
1752799180cyclictest0-21swapper/1120:09:483
17526991817cyclictest831ksoftirqd/1022:55:142
17526991817cyclictest0-21swapper/1019:56:072
1752699181cyclictest831ksoftirqd/1019:45:142
1752699181cyclictest831ksoftirqd/1019:40:242
1752699181cyclictest811rcuc/1019:20:002
1752699181cyclictest188622chrt19:50:562
1752599181cyclictest4208-1kworker/9:0H20:10:3211
1752599181cyclictest12250irq/42-ahci21:47:1811
1752599181cyclictest0-21swapper/923:26:3411
1752599181cyclictest0-21swapper/923:26:3411
1752599181cyclictest0-21swapper/923:03:4811
1752599181cyclictest0-21swapper/922:30:2211
1752599181cyclictest0-21swapper/921:10:1011
1752399186cyclictest0-21swapper/723:05:019
17522991816cyclictest20533-21qemu-kvm22:19:438
1748599180cyclictest0-21swapper/019:26:390
17526991716cyclictest831ksoftirqd/1023:10:142
17526991716cyclictest831ksoftirqd/1021:00:152
17526991716cyclictest831ksoftirqd/1020:00:362
17526991716cyclictest811rcuc/1023:45:082
17526991716cyclictest0-21swapper/1023:40:212
1752699170cyclictest831ksoftirqd/1023:25:152
1752699170cyclictest831ksoftirqd/1023:25:152
1752699170cyclictest831ksoftirqd/1022:50:142
1752699170cyclictest831ksoftirqd/1022:40:122
1752699170cyclictest831ksoftirqd/1022:35:162
1752699170cyclictest831ksoftirqd/1022:05:142
1752699170cyclictest831ksoftirqd/1021:20:122
1752699170cyclictest831ksoftirqd/1020:45:312
1752699170cyclictest831ksoftirqd/1020:20:022
1752699170cyclictest831ksoftirqd/1020:05:132
1752699170cyclictest831ksoftirqd/1019:30:162
1752699170cyclictest831ksoftirqd/1000:20:022
1752699170cyclictest811rcuc/1021:50:222
1752699170cyclictest811rcuc/1020:35:382
1752699170cyclictest811rcuc/1020:35:372
1752699170cyclictest74652sleep1023:20:072
1752699170cyclictest70962sleep1000:01:362
1752699170cyclictest63312sleep1022:32:262
1752699170cyclictest57372sleep1021:46:212
1752699170cyclictest4592sleep1023:55:112
1752699170cyclictest38352sleep1023:15:062
1752699170cyclictest325952sleep1000:37:152
1752699170cyclictest322842chrt23:53:232
1752699170cyclictest322842chrt23:53:232
1752699170cyclictest319612sleep1022:25:012
1752699170cyclictest310742sleep1019:25:182
1752699170cyclictest310412chrt21:39:322
1752699170cyclictest306842sleep1020:10:082
1752699170cyclictest3066350irq/52-eth0-rx-22:21:102
1752699170cyclictest3066350irq/52-eth0-rx-20:15:292
1752699170cyclictest3066350irq/52-eth0-rx-20:15:292
1752699170cyclictest3066350irq/52-eth0-rx-19:35:222
1752699170cyclictest3066350irq/52-eth0-rx-00:25:282
1752699170cyclictest289602sleep1000:32:152
1752699170cyclictest289152sleep1023:05:062
1752699170cyclictest281272sleep1023:01:162
1752699170cyclictest274542sleep1020:50:022
1752699170cyclictest274542sleep1020:50:022
1752699170cyclictest248402chrt22:15:122
1752699170cyclictest241652sleep1022:13:002
1752699170cyclictest235032sleep1021:25:422
1752699170cyclictest225112sleep1019:15:152
1752699170cyclictest219852sleep1020:40:252
1752699170cyclictest219852sleep1020:40:252
1752699170cyclictest21802sleep1021:43:332
1752699170cyclictest199472sleep1019:10:272
1752699170cyclictest157632sleep1020:31:362
1752699170cyclictest138222sleep1022:00:102
1752699170cyclictest126012sleep1021:11:172
1752699170cyclictest123542sleep1020:29:472
1752699170cyclictest12250irq/42-ahci22:45:332
1752699170cyclictest12250irq/42-ahci21:15:092
1752699170cyclictest12250irq/42-ahci21:08:362
1752699170cyclictest115072chrt00:10:112
1752699170cyclictest101762sleep1021:55:092
17525991716cyclictest0-21swapper/922:50:1211
17525991715cyclictest27308-21chrt21:33:0611
17525991715cyclictest16415-21taskset21:18:4011
1752599170cyclictest4208-1kworker/9:0H20:09:4211
1752599170cyclictest24381-21awk23:40:2811
1752399175cyclictest0-21swapper/720:25:019
17522991716cyclictest30015-21qemu-kvm20:10:518
17522991715cyclictest30016-21qemu-kvm20:08:598
17522991715cyclictest17246-21qemu-kvm19:25:418
17518991716cyclictest17245-21qemu-kvm19:12:437
17512991715cyclictest22906-21sensors21:25:316
1752799160cyclictest0-21swapper/1120:10:023
17525991615cyclictest4208-1kworker/9:0H19:25:2011
17525991615cyclictest3066350irq/52-eth0-rx-23:16:5711
17525991615cyclictest3066350irq/52-eth0-rx-19:39:3611
17525991615cyclictest2539-1kworker/9:2H23:52:2111
17525991615cyclictest2539-1kworker/9:2H23:52:2111
17525991615cyclictest0-21swapper/923:20:1411
17525991615cyclictest0-21swapper/920:16:4611
17525991615cyclictest0-21swapper/920:16:4611
17525991614cyclictest28755-21snmp_rack3slot922:20:1211
17523991614cyclictest8329-21diskstats23:20:179
17522991615cyclictest20532-21qemu-kvm22:08:398
17522991614cyclictest17246-21qemu-kvm19:18:348
17518991615cyclictest8744-21python21:50:297
1750599161cyclictest0-21swapper/321:29:345
17485991615cyclictest0-21swapper/021:45:270
1748599160cyclictest0-21swapper/021:17:190
1752799154cyclictest0-21swapper/1120:57:323
17527991513cyclictest27701-21chrt22:16:373
1752599155cyclictest0-21swapper/920:40:0111
1752599155cyclictest0-21swapper/920:40:0011
1752599155cyclictest0-21swapper/900:30:0211
17525991514cyclictest2539-1kworker/9:2H22:47:3311
17525991514cyclictest0-21swapper/923:10:1611
17525991514cyclictest0-21swapper/922:35:4411
17525991514cyclictest0-21swapper/900:35:1811
17525991513cyclictest12381-21munin-run20:30:0011
1752399154cyclictest0-21swapper/720:05:219
1752399153cyclictest0-21swapper/722:29:599
1752299154cyclictest0-21swapper/623:54:198
1752299154cyclictest0-21swapper/623:54:198
17518991514cyclictest14615-21latency_hist00:15:017
17512991513cyclictest26537-21sensors21:30:266
1750599155cyclictest0-21swapper/323:51:455
1750599155cyclictest0-21swapper/323:51:455
1750599155cyclictest0-21swapper/321:53:565
1750599155cyclictest0-21swapper/320:22:125
1750599155cyclictest0-21swapper/320:11:345
1750599150cyclictest0-21swapper/321:17:315
6699140watchdog/80-21swapper/821:12:3510
17525991413cyclictest741rcuc/920:47:1311
17525991413cyclictest5581-21kworker/9:119:10:1811
17525991413cyclictest31979-21chrt23:09:1711
17525991413cyclictest3066350irq/52-eth0-rx-19:18:0611
17525991413cyclictest0-21swapper/921:20:1211
1752399144cyclictest0-21swapper/723:50:019
1752399144cyclictest0-21swapper/722:20:019
1752399144cyclictest0-21swapper/722:10:019
1752399144cyclictest0-21swapper/700:10:019
1752399143cyclictest0-21swapper/723:40:019
1752299145cyclictest0-21swapper/620:30:218
17522991413cyclictest30015-21qemu-kvm20:47:248
17522991412cyclictest17246-21qemu-kvm19:24:208
1750599144cyclictest0-21swapper/320:43:315
1750599144cyclictest0-21swapper/320:43:305
1748599140cyclictest0-21swapper/021:08:160
6699130watchdog/80-21swapper/822:15:1510
1752799133cyclictest0-21swapper/1123:35:483
1752799133cyclictest0-21swapper/1123:35:483
1752799133cyclictest0-21swapper/1122:20:013
1752799133cyclictest0-21swapper/1121:50:013
17527991312cyclictest235312chrt21:26:063
17526991312cyclictest831ksoftirqd/1000:05:212
17526991312cyclictest0-21swapper/1000:20:132
1752599133cyclictest0-21swapper/923:45:0111
1752599133cyclictest0-21swapper/920:20:0211
1752599133cyclictest0-21swapper/919:53:1511
1752599133cyclictest0-21swapper/900:25:0211
1752599133cyclictest0-21swapper/900:17:3011
17525991312cyclictest4208-1kworker/9:0H21:00:2911
17525991312cyclictest3066350irq/52-eth0-rx-22:57:3711
17525991312cyclictest27945-21cat22:19:5511
17525991312cyclictest0-21swapper/921:56:1711
17525991312cyclictest0-21swapper/900:10:1611
1752399134cyclictest0-21swapper/723:50:019
1752399134cyclictest0-21swapper/723:50:009
1752399134cyclictest0-21swapper/719:19:599
1752399133cyclictest0-21swapper/722:51:539
1752399133cyclictest0-21swapper/721:01:309
1752399133cyclictest0-21swapper/720:12:079
1752399133cyclictest0-21swapper/719:32:069
1752399133cyclictest0-21swapper/719:25:369
17523991310cyclictest621ksoftirqd/700:10:199
1752299135cyclictest0-21swapper/623:39:488
1752299135cyclictest0-21swapper/623:39:488
1752299135cyclictest0-21swapper/622:55:128
1752299135cyclictest0-21swapper/600:24:498
1752299134cyclictest0-21swapper/619:30:008
1752299133cyclictest0-21swapper/623:46:448
1752299133cyclictest0-21swapper/623:25:468
1752299133cyclictest0-21swapper/623:25:468
1752299133cyclictest0-21swapper/620:43:098
1752299133cyclictest0-21swapper/620:43:088
1752299133cyclictest0-21swapper/620:00:008
1752299132cyclictest0-21swapper/619:55:018
1752299131cyclictest0-21swapper/620:00:018
17518991312cyclictest15516-21latency_hist19:50:017
1750599133cyclictest0-21swapper/321:10:185
1750599130cyclictest0-21swapper/323:46:245
17489991312cyclictest3066350irq/52-eth0-rx-19:30:391
17485991311cyclictest20973-21taskset19:13:170
1748599131cyclictest0-21swapper/023:20:280
901120ksoftirqd/110-21swapper/1119:20:313
6699120watchdog/80-21swapper/800:29:4610
411120ksoftirqd/40-21swapper/419:55:506
411120ksoftirqd/40-21swapper/419:20:306
3199120watchdog/30-21swapper/323:57:195
271120ksoftirqd/20-21swapper/219:20:314
1752799123cyclictest0-21swapper/1123:24:593
1752799123cyclictest0-21swapper/1122:00:003
1752799123cyclictest0-21swapper/1119:55:013
1752799123cyclictest0-21swapper/1100:40:013
1752799122cyclictest0-21swapper/1119:36:023
1752799122cyclictest0-21swapper/1100:10:133
1752599123cyclictest0-21swapper/921:27:0011
17525991211cyclictest9802-21needreboot00:05:2511
17525991211cyclictest0-21swapper/921:05:1511
17525991211cyclictest0-21swapper/919:25:0111
17525991210cyclictest1618-21perl23:55:1511
1752399124cyclictest0-21swapper/721:35:009
1752399124cyclictest0-21swapper/721:30:019
1752399124cyclictest0-21swapper/721:13:339
1752399124cyclictest0-21swapper/720:55:019
1752399124cyclictest0-21swapper/720:55:019
1752399124cyclictest0-21swapper/720:30:019
1752399123cyclictest0-21swapper/720:27:399
1752399122cyclictest0-21swapper/723:57:169
1752399122cyclictest0-21swapper/723:16:159
1752399122cyclictest0-21swapper/721:08:479
1752399122cyclictest0-21swapper/719:40:019
1752399122cyclictest0-21swapper/719:22:369
1752399121cyclictest0-21swapper/723:30:029
1752299122cyclictest0-21swapper/622:51:298
1752299120cyclictest0-21swapper/619:45:008
17518991211cyclictest930-21cstates23:55:127
1750599128cyclictest0-21swapper/323:37:245
1750599128cyclictest0-21swapper/323:37:235
1750599128cyclictest0-21swapper/323:03:025
1750599128cyclictest0-21swapper/321:49:205
1750599128cyclictest0-21swapper/321:41:405
1750599128cyclictest0-21swapper/321:03:545
1750599128cyclictest0-21swapper/320:48:585
1750599128cyclictest0-21swapper/320:09:555
1750599128cyclictest0-21swapper/300:33:505
1750599128cyclictest0-21swapper/300:23:105
1750599128cyclictest0-21swapper/300:10:505
1750599128cyclictest0-21swapper/300:02:525
1750599126cyclictest0-21swapper/321:09:105
1750599125cyclictest0-21swapper/300:18:255
1750599124cyclictest0-21swapper/323:34:405
1750599121cyclictest0-21swapper/323:40:085
1750599121cyclictest0-21swapper/323:28:485
1750599121cyclictest0-21swapper/323:28:485
1750599121cyclictest0-21swapper/323:21:365
1750599121cyclictest0-21swapper/323:18:005
1750599121cyclictest0-21swapper/323:12:235
1750599121cyclictest0-21swapper/323:07:085
1750599121cyclictest0-21swapper/321:55:565
1750599121cyclictest0-21swapper/321:36:005
1750599121cyclictest0-21swapper/321:31:385
1750599121cyclictest0-21swapper/320:58:245
1750599121cyclictest0-21swapper/320:54:565
1750599121cyclictest0-21swapper/320:54:565
1750599121cyclictest0-21swapper/320:25:405
1750599121cyclictest0-21swapper/320:01:445
1750599121cyclictest0-21swapper/300:36:045
1750599121cyclictest0-21swapper/300:27:425
1750599121cyclictest0-21swapper/300:09:395
17489991211cyclictest13709-21python00:10:301
1748999120cyclictest0-21swapper/122:25:171
1748599123cyclictest0-21swapper/021:25:170
17485991210cyclictest4392chrt20:10:350
17485991210cyclictest2419-21sshd22:26:440
1748599120cyclictest0-21swapper/020:55:290
1748599120cyclictest0-21swapper/020:30:420
1699120watchdog/00-21swapper/022:33:190
691110ksoftirqd/80-21swapper/819:20:3010
6899110migration/814977-21munin-node21:15:1910
6699110watchdog/80-21swapper/820:23:3810
621110ksoftirqd/70-21swapper/719:55:509
5499110migration/628322-21munin-node20:50:158
5499110migration/628322-21munin-node20:50:158
5499110migration/616259-21needreboot22:45:228
481110ksoftirqd/50-21swapper/519:55:497
4599110watchdog/50-21swapper/522:56:587
341110ksoftirqd/30-21swapper/319:20:315
3066450110irq/53-eth0-tx-0-21swapper/919:45:1411
31110ksoftirqd/00-21swapper/019:55:500
31110ksoftirqd/00-21swapper/019:30:280
201110ksoftirqd/10-21swapper/119:55:501
1752799114cyclictest0-21swapper/1123:53:443
1752799114cyclictest0-21swapper/1123:53:443
1752799114cyclictest0-21swapper/1121:05:013
1752799114cyclictest0-21swapper/1119:55:013
1752799113cyclictest0-21swapper/1123:00:133
1752799113cyclictest0-21swapper/1122:36:173
1752799113cyclictest0-21swapper/1121:55:063
1752799113cyclictest0-21swapper/1121:43:523
1752799113cyclictest0-21swapper/1120:34:523
1752799113cyclictest0-21swapper/1120:02:223
1752799113cyclictest0-21swapper/1100:17:463
1752799111cyclictest0-21swapper/1100:25:013
1752599113cyclictest0-21swapper/923:35:0111
1752599113cyclictest0-21swapper/923:35:0111
17525991110cyclictest0-21swapper/922:10:3311
17525991110cyclictest0-21swapper/920:00:1611
1752599111cyclictest0-21swapper/919:56:2911
1752399115cyclictest0-21swapper/723:27:489
1752399115cyclictest0-21swapper/723:27:479
1752399115cyclictest0-21swapper/721:51:079
1752399113cyclictest0-21swapper/723:36:159
1752399113cyclictest0-21swapper/723:36:159
1752399113cyclictest0-21swapper/722:59:149
1752399113cyclictest0-21swapper/722:17:529
1752399113cyclictest0-21swapper/722:14:439
1752399113cyclictest0-21swapper/721:20:409
1752399113cyclictest0-21swapper/700:30:459
1752399111cyclictest0-21swapper/700:19:479
1752299119cyclictest20532-21qemu-kvm22:23:178
1752299115cyclictest0-21swapper/622:00:478
1752299114cyclictest0-21swapper/623:23:438
1752299113cyclictest0-21swapper/622:31:138
1752299113cyclictest0-21swapper/621:47:128
1752299113cyclictest0-21swapper/621:40:038
1752299113cyclictest0-21swapper/621:36:118
1752299113cyclictest0-21swapper/621:32:378
1752299113cyclictest0-21swapper/600:36:168
1752299113cyclictest0-21swapper/600:33:478
1752299113cyclictest0-21swapper/600:08:468
1752299113cyclictest0-21swapper/600:01:308
17522991110cyclictest30015-21qemu-kvm20:27:218
1751899113cyclictest0-21swapper/523:45:177
1751899113cyclictest0-21swapper/500:10:017
17518991110cyclictest17246-21qemu-kvm19:36:477
1751299113cyclictest0-21swapper/420:55:196
1750599115cyclictest0-21swapper/322:00:465
1750599115cyclictest0-21swapper/321:20:485
1750599115cyclictest0-21swapper/320:36:585
1750599115cyclictest0-21swapper/320:36:575
1750599115cyclictest0-21swapper/320:31:085
1750599115cyclictest0-21swapper/320:15:085
1750599115cyclictest0-21swapper/320:15:085
1750599111cyclictest0-21swapper/322:48:145
1748599118cyclictest0-21swapper/020:50:300
1748599118cyclictest0-21swapper/020:50:300
1748599118cyclictest0-21swapper/020:15:330
1748599118cyclictest0-21swapper/020:15:320
1748599113cyclictest0-21swapper/022:09:440
1748599113cyclictest0-21swapper/020:28:410
1748599113cyclictest0-21swapper/020:20:350
1748599113cyclictest0-21swapper/000:30:180
1748599113cyclictest0-21swapper/000:25:210
1748599110cyclictest0-21swapper/022:40:260
14850110irq/18-uhci_hcd0-21swapper/023:40:250
14850110irq/18-uhci_hcd0-21swapper/021:00:310
14850110irq/18-uhci_hcd0-21swapper/000:00:260
901100ksoftirqd/110-21swapper/1123:40:253
901100ksoftirqd/110-21swapper/1122:25:133
901100ksoftirqd/110-21swapper/1121:35:083
901100ksoftirqd/110-21swapper/1121:00:373
901100ksoftirqd/110-21swapper/1119:15:353
8999100migration/116506-21switchtime00:00:303
8999100migration/1112794-21taskset21:13:563
8799100watchdog/110-21swapper/1120:45:233
7399100watchdog/90-21swapper/922:01:1111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional