You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 19:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sun Nov 23, 2025 12:46:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501140irq/42-ahci0-21swapper/107:09:071
24352211095sleep90-21swapper/907:05:1711
22990211096sleep80-21swapper/807:05:0410
25853210895sleep40-21swapper/407:06:366
25892210795sleep70-21swapper/707:07:049
25881210490sleep100-21swapper/1007:06:542
25961210290sleep50-21swapper/507:08:027
3066350960irq/52-eth0-rx-0-21swapper/307:07:495
3066350960irq/52-eth0-rx-0-21swapper/1107:05:133
3066450930irq/53-eth0-tx-0-21swapper/207:05:234
2594926353sleep60-21swapper/607:07:518
2299525955sleep00-21swapper/007:05:080
26264992827cyclictest0-21swapper/007:26:390
26307991917cyclictest0-21swapper/1107:26:533
2630699192cyclictest3066350irq/52-eth0-rx-11:40:302
2626499190cyclictest0-21swapper/010:35:260
26306991817cyclictest0-21swapper/1008:31:362
26306991812cyclictest32524-21processes10:55:262
26306991812cyclictest32524-21processes10:55:262
2630699181cyclictest3066350irq/52-eth0-rx-11:05:212
2630699181cyclictest3066350irq/52-eth0-rx-09:10:312
2630699181cyclictest241722chrt07:48:222
2630699181cyclictest12250irq/42-ahci09:08:552
2630699181cyclictest12250irq/42-ahci09:08:552
2630699180cyclictest155502chrt11:17:212
2630699180cyclictest12250irq/42-ahci07:21:552
2630599181cyclictest761ksoftirqd/909:05:2111
2630599181cyclictest761ksoftirqd/909:05:2111
2630599181cyclictest761ksoftirqd/907:40:0111
2630599181cyclictest0-21swapper/908:55:1311
26304991816cyclictest8672-21perl08:55:1510
26304991816cyclictest736-21snmp_easybox.os12:25:2210
26304991816cyclictest5215-21memory08:05:2110
26304991816cyclictest4304-21snmp_easybox.os12:30:2610
26304991816cyclictest4086-21snmp_easybox.os11:45:2410
26304991816cyclictest3908-21snmp_easybox.os08:50:1110
26304991816cyclictest32613-21snmp_easybox.os08:45:1210
26304991816cyclictest28029-21diskstats11:35:1510
26304991816cyclictest27716-21snmp_rack3slot907:51:5010
26304991816cyclictest27524-21snmp_rack3slot907:10:1410
26304991816cyclictest24315-21perl10:45:1410
26304991816cyclictest24315-21perl10:45:1410
26304991816cyclictest21320-21users09:10:3110
26304991816cyclictest20348-21snmp_easybox.os12:10:1310
26304991816cyclictest18496-21snmp_rack3slot908:25:1310
26304991816cyclictest17879-21perl12:05:1510
26304991816cyclictest17019-21munin-run07:40:0010
26304991816cyclictest1484-21munin-run11:45:0110
26304991816cyclictest14091-21perl12:00:1610
26304991816cyclictest1312-21snmp_easybox.os07:15:2810
26304991816cyclictest10360-21memory10:25:2110
26302991816cyclictest26023-21qemu-kvm07:30:418
26302991816cyclictest17882-21qemu-kvm09:17:108
2626499180cyclictest0-21swapper/012:20:270
2626499180cyclictest0-21swapper/007:15:300
2626499180cyclictest0-21swapper/007:10:260
26306991716cyclictest28488-21python10:05:252
2630699170cyclictest92682sleep1008:10:242
2630699170cyclictest92682sleep1008:10:232
2630699170cyclictest92072sleep1012:39:552
2630699170cyclictest87122chrt11:53:262
2630699170cyclictest72682sleep1009:36:342
2630699170cyclictest68082sleep1008:52:062
2630699170cyclictest63582sleep1008:07:362
2630699170cyclictest52822sleep1012:30:552
2630699170cyclictest49782sleep1011:47:012
2630699170cyclictest47732sleep1011:04:092
2630699170cyclictest47732sleep1011:04:092
2630699170cyclictest41272sleep1010:16:282
2630699170cyclictest319872chrt08:43:412
2630699170cyclictest312492chrt07:55:352
2630699170cyclictest311892sleep1012:25:072
2630699170cyclictest3066450irq/53-eth0-tx-11:20:132
2630699170cyclictest3066450irq/53-eth0-tx-08:25:152
2630699170cyclictest3066350irq/52-eth0-rx-11:10:372
2630699170cyclictest3066350irq/52-eth0-rx-10:30:552
2630699170cyclictest3066350irq/52-eth0-rx-09:25:312
2630699170cyclictest304472chrt12:21:532
2630699170cyclictest299732sleep1010:10:022
2630699170cyclictest298392sleep1007:14:292
2630699170cyclictest296082chrt10:51:192
2630699170cyclictest271712sleep1011:35:132
2630699170cyclictest268432sleep1007:50:222
2630699170cyclictest265052sleep1011:33:002
2630699170cyclictest260862sleep1010:48:062
2630699170cyclictest260862sleep1010:48:062
2630699170cyclictest258242sleep1009:20:092
2630699170cyclictest254742sleep1010:01:132
2630699170cyclictest253102sleep1008:35:032
2630699170cyclictest25252sleep1009:30:222
2630699170cyclictest250902sleep1009:17:082
2630699170cyclictest226912sleep1011:25:482
2630699170cyclictest223262chrt10:41:302
2630699170cyclictest219362sleep1009:58:012
2630699170cyclictest211562sleep1012:10:152
2630699170cyclictest205342sleep1007:43:342
2630699170cyclictest194822sleep1012:06:152
2630699170cyclictest189332sleep1010:39:542
2630699170cyclictest184822sleep1009:55:002
2630699170cyclictest172342sleep1008:21:032
2630699170cyclictest167902sleep1007:36:572
2630699170cyclictest160272chrt12:03:392
2630699170cyclictest15122sleep1008:45:172
2630699170cyclictest142182sleep1009:03:302
2630699170cyclictest137582chrt08:18:262
2630699170cyclictest12250irq/42-ahci09:45:192
2630699170cyclictest12250irq/42-ahci07:15:112
2630699170cyclictest119112sleep1007:30:212
2630699170cyclictest105932sleep1008:59:062
2630699170cyclictest103902sleep1011:55:142
26305991716cyclictest761ksoftirqd/910:25:2711
26305991716cyclictest0-21swapper/908:20:3011
2630599170cyclictest16077-21expr10:35:1411
2630599170cyclictest0-21swapper/911:45:2211
2630599170cyclictest0-21swapper/911:15:1511
2630599170cyclictest0-21swapper/910:55:2211
2630599170cyclictest0-21swapper/910:55:2211
2630599170cyclictest0-21swapper/910:40:2111
2630599170cyclictest0-21swapper/909:00:1411
2630599170cyclictest0-21swapper/908:30:1111
2630599170cyclictest0-21swapper/907:40:0011
2630599170cyclictest0-21swapper/907:25:1911
26304991715cyclictest9211-21perl09:40:1510
26304991715cyclictest8967-21snmp_rack3slot912:36:3710
26304991715cyclictest7975-21snmp_rack3slot908:10:1410
26304991715cyclictest7975-21snmp_rack3slot908:10:1410
26304991715cyclictest7480-21diskstats07:25:1310
26304991715cyclictest3758-21snmp_rack3slot907:20:1210
26304991715cyclictest32120-21snmp_easybox.os08:00:1110
26304991715cyclictest31987-21iostat10:55:2010
26304991715cyclictest31987-21iostat10:55:2010
26304991715cyclictest30203-21snmp_rack3slot909:25:1410
26304991715cyclictest30047-21snmp_easybox.os10:10:1410
26304991715cyclictest29714-21perl07:55:1710
26304991715cyclictest28747-21perl12:20:1610
26304991715cyclictest23473-21diskstats10:00:1310
26304991715cyclictest22545-21snmp_rack3slot908:30:1410
26304991715cyclictest22214-21snmp_easybox.os09:15:1110
26304991715cyclictest18211-21snmp_rack3slot909:51:4210
26304991715cyclictest17471-21perl11:20:1410
26304991715cyclictest16441-21iostat_ios09:05:1910
26304991715cyclictest16441-21iostat_ios09:05:1810
26304991715cyclictest13549-21snmp_rack3slot910:30:1510
26304991715cyclictest10097-21perl11:10:1510
2630299172cyclictest0-21swapper/607:22:158
26279991715cyclictest2406-21snmpd10:15:524
2626499170cyclictest0-21swapper/008:15:270
2630799164cyclictest0-21swapper/1112:00:023
2630799164cyclictest0-21swapper/1110:55:013
2630799164cyclictest0-21swapper/1110:55:013
2630799160cyclictest0-21swapper/1107:12:253
26305991615cyclictest761ksoftirqd/912:35:3611
26305991615cyclictest761ksoftirqd/907:50:1711
26305991615cyclictest2976-21chrt07:20:0611
26305991615cyclictest10458-21latency_hist08:15:0111
26305991615cyclictest10458-21latency_hist08:15:0011
26304991615cyclictest12289-21ntp_states07:30:2210
26304991614cyclictest6568-21perl11:05:1510
26304991614cyclictest29745-21df08:40:1310
26304991614cyclictest29467-21munin-run10:10:0110
26304991614cyclictest26421-21snmp_rack3slot911:31:5010
26304991614cyclictest2421-21df11:00:1410
26304991614cyclictest2421-21df11:00:1410
26304991614cyclictest21238-21snmp_rack3slot907:45:1310
26304991614cyclictest18903-21snmp_rack3slot907:40:1810
26304991614cyclictest17090-21snmp_rack3slot910:35:1610
26304991614cyclictest14416-21snmp_easybox.os08:20:1310
26304991614cyclictest1233-21snmp_easybox.os10:15:1410
26304991614cyclictest12044-21diskstats09:00:1310
2630299165cyclictest24308-21idleruntime-cro07:50:018
2630299163cyclictest0-21swapper/608:32:068
2630299163cyclictest0-21swapper/607:36:108
26302991614cyclictest17881-21qemu-kvm09:08:378
26302991614cyclictest17881-21qemu-kvm09:08:378
2629999162cyclictest0-21swapper/509:30:187
26299991615cyclictest4369-21cstates08:50:117
26299991615cyclictest26022-21qemu-kvm07:10:357
26299991615cyclictest17882-21qemu-kvm09:24:027
26299991615cyclictest13905-21python09:45:347
26299991614cyclictest29762-21python12:20:287
2628599160cyclictest0-21swapper/309:40:215
2628599160cyclictest0-21swapper/307:15:225
26279991614cyclictest2406-21snmpd11:18:084
2626899160cyclictest0-21swapper/107:30:201
2626899160cyclictest0-21swapper/107:20:291
2626499160cyclictest0-21swapper/012:10:270
2626499160cyclictest0-21swapper/012:00:240
2626499160cyclictest0-21swapper/009:15:250
26307991514cyclictest12250irq/42-ahci07:21:273
26307991514cyclictest116322sleep1110:29:293
26306991514cyclictest10544-1kworker/10:2H10:25:342
2630599154cyclictest0-21swapper/910:20:0111
26305991514cyclictest9236-21perl09:40:1511
26305991514cyclictest0-21swapper/911:27:3811
26305991514cyclictest0-21swapper/909:15:1611
26305991514cyclictest0-21swapper/907:10:2311
2630599151cyclictest741rcuc/912:20:1311
26304991513cyclictest6827-21perl11:50:1510
26304991513cyclictest12136-21snmp_rack3slot911:55:3610
26293991513cyclictest2489-21sendmail10:00:186
2629399150cyclictest0-21swapper/407:15:226
2628599150cyclictest0-21swapper/309:45:365
2627999150cyclictest0-21swapper/209:45:184
2627999150cyclictest0-21swapper/208:50:014
26268991514cyclictest3066350irq/52-eth0-rx-09:15:131
2626499156cyclictest0-21swapper/008:04:300
2626499152cyclictest0-21swapper/010:08:270
2626499150cyclictest21168-21munin-node07:45:210
2626499150cyclictest0-21swapper/010:40:170
2626499150cyclictest0-21swapper/010:20:130
2626499150cyclictest0-21swapper/009:40:160
2626499150cyclictest0-21swapper/009:30:240
2626499150cyclictest0-21swapper/009:20:250
2626499150cyclictest0-21swapper/008:50:260
2626499150cyclictest0-21swapper/008:45:070
2626499150cyclictest0-21swapper/008:35:240
2626499150cyclictest0-21swapper/008:06:490
2630599144cyclictest0-21swapper/912:10:0011
2630599142cyclictest0-21swapper/911:35:1411
26305991413cyclictest0-21swapper/912:15:2111
26305991413cyclictest0-21swapper/912:03:2811
26305991413cyclictest0-21swapper/909:55:1511
26305991413cyclictest0-21swapper/908:40:1811
26305991413cyclictest0-21swapper/907:30:1211
26305991412cyclictest13162-21snmp_rack3slot910:30:1411
2630599141cyclictest0-21swapper/909:30:0011
2630499144cyclictest0-21swapper/808:40:0110
26304991412cyclictest25498-21iostat12:15:1910
26304991412cyclictest22772-21snmp_rack3slot911:26:4910
26304991412cyclictest12553-21df_inode09:45:1910
2630399145cyclictest0-21swapper/712:15:009
2630299143cyclictest0-21swapper/607:29:368
26293991413cyclictest5178-21sensors07:20:236
26279991412cyclictest2406-21snmpd09:24:514
2626899144cyclictest0-21swapper/110:15:011
2626499148cyclictest0-21swapper/011:35:260
2626499148cyclictest0-21swapper/011:10:270
2626499147cyclictest0-21swapper/011:05:160
2626499147cyclictest0-21swapper/009:10:260
2626499146cyclictest0-21swapper/011:26:080
2626499145cyclictest0-21swapper/012:05:350
2626499145cyclictest0-21swapper/009:59:040
2626499145cyclictest0-21swapper/008:33:200
26264991413cyclictest0-21swapper/012:35:480
2626499141cyclictest0-21swapper/010:35:010
2626499141cyclictest0-21swapper/010:25:490
2626499140cyclictest10663-21latency_hist09:00:020
2626499140cyclictest0-21swapper/012:15:270
2626499140cyclictest0-21swapper/011:56:450
2626499140cyclictest0-21swapper/011:50:260
2626499140cyclictest0-21swapper/011:45:260
2626499140cyclictest0-21swapper/011:35:010
2626499140cyclictest0-21swapper/011:15:270
2626499140cyclictest0-21swapper/010:59:200
2626499140cyclictest0-21swapper/010:59:200
2626499140cyclictest0-21swapper/010:11:210
2626499140cyclictest0-21swapper/009:49:260
2626499140cyclictest0-21swapper/008:56:420
2626499140cyclictest0-21swapper/008:42:330
2626499140cyclictest0-21swapper/008:28:000
2626499140cyclictest0-21swapper/008:10:210
2626499140cyclictest0-21swapper/008:10:210
2626499140cyclictest0-21swapper/007:55:270
2626499140cyclictest0-21swapper/007:30:250
2630799134cyclictest0-21swapper/1110:15:223
2630799133cyclictest0-21swapper/1111:15:383
2630699134cyclictest0-21swapper/1008:00:012
2630599133cyclictest0-21swapper/911:34:4711
2630599133cyclictest0-21swapper/911:10:4211
2630599133cyclictest0-21swapper/910:10:0111
2630599133cyclictest0-21swapper/909:35:4411
2630599133cyclictest0-21swapper/908:05:3111
26305991312cyclictest8221-21snmp_rack3slot911:06:3711
26305991312cyclictest761ksoftirqd/912:00:0211
26305991312cyclictest761ksoftirqd/911:50:1411
26305991312cyclictest761ksoftirqd/910:05:2111
26305991312cyclictest761ksoftirqd/909:20:1111
26305991312cyclictest32256-21cat08:00:1111
26305991312cyclictest22305-21grep07:45:1611
26305991312cyclictest0-21swapper/911:00:5811
26305991312cyclictest0-21swapper/911:00:5811
26305991312cyclictest0-21swapper/910:54:3911
26305991312cyclictest0-21swapper/909:45:2811
26305991312cyclictest0-21swapper/908:47:5411
26305991312cyclictest0-21swapper/908:35:0211
26305991312cyclictest0-21swapper/908:28:3811
26305991312cyclictest0-21swapper/908:15:2711
26305991312cyclictest0-21swapper/907:15:1611
26305991311cyclictest25490-21chrt10:01:2411
2630499133cyclictest0-21swapper/809:35:0010
26304991311cyclictest29554-21snmp_rack3slot910:50:3710
26304991311cyclictest12633-21snmp_easybox.os11:15:1210
2630399133cyclictest0-21swapper/712:35:009
2630399133cyclictest0-21swapper/712:07:479
2630399133cyclictest0-21swapper/711:56:029
2630399133cyclictest0-21swapper/711:20:479
2630399133cyclictest0-21swapper/710:05:019
2630399133cyclictest0-21swapper/708:20:519
2630299133cyclictest0-21swapper/612:26:328
2630299133cyclictest0-21swapper/612:13:538
2630299133cyclictest0-21swapper/611:31:238
2630299132cyclictest0-21swapper/607:14:408
26299991312cyclictest26023-21qemu-kvm07:47:297
26299991312cyclictest24358-21cpuspeed_turbos12:15:127
26293991312cyclictest18451-21sensors11:20:256
2629399130cyclictest0-21swapper/409:40:216
2627999134cyclictest0-21swapper/209:10:174
2627999133cyclictest0-21swapper/209:15:314
2627999132cyclictest0-21swapper/211:45:154
2627999132cyclictest0-21swapper/210:50:314
26279991311cyclictest2406-21snmpd10:40:374
26279991311cyclictest2406-21snmpd08:38:204
26268991312cyclictest3066350irq/52-eth0-rx-10:59:181
26268991312cyclictest3066350irq/52-eth0-rx-10:59:181
2626499137cyclictest0-21swapper/012:25:240
2626499136cyclictest0-21swapper/009:50:130
2626499136cyclictest0-21swapper/009:37:590
2626499136cyclictest0-21swapper/009:08:270
2626499136cyclictest0-21swapper/009:08:270
2626499136cyclictest0-21swapper/008:20:510
2626499136cyclictest0-21swapper/007:20:230
2626499135cyclictest0-21swapper/010:50:590
2626499134cyclictest0-21swapper/012:30:140
2626499130cyclictest0-21swapper/011:40:050
2626499130cyclictest0-21swapper/011:00:080
2626499130cyclictest0-21swapper/011:00:080
2626499130cyclictest0-21swapper/010:46:090
2626499130cyclictest0-21swapper/010:46:090
2626499130cyclictest0-21swapper/009:30:000
2626499130cyclictest0-21swapper/007:35:140
411120ksoftirqd/40-21swapper/408:05:016
2630799123cyclictest0-21swapper/1112:20:093
2630799122cyclictest0-21swapper/1108:45:023
2630799122cyclictest0-21swapper/1107:40:563
2630599123cyclictest0-21swapper/912:09:5911
2630599123cyclictest0-21swapper/907:55:0111
2630599122cyclictest0-21swapper/912:34:3711
2630599122cyclictest0-21swapper/909:32:1411
26305991211cyclictest18189-21snmp_rack3slot909:51:3011
26305991211cyclictest0-21swapper/910:17:5011
26304991210cyclictest27097-21perl09:20:1410
2630399122cyclictest0-21swapper/712:20:539
2630399122cyclictest0-21swapper/711:12:239
2630399122cyclictest0-21swapper/709:57:219
2630399122cyclictest0-21swapper/709:50:369
2630399122cyclictest0-21swapper/709:02:059
2630399122cyclictest0-21swapper/708:53:019
2630399122cyclictest0-21swapper/707:26:109
2630399121cyclictest0-21swapper/707:20:009
2630399120cyclictest0-21swapper/708:34:569
2630299123cyclictest0-21swapper/611:29:188
2630299122cyclictest0-21swapper/612:39:248
2630299122cyclictest0-21swapper/611:44:538
2630299122cyclictest0-21swapper/610:54:018
2630299121cyclictest0-21swapper/610:40:208
2629999123cyclictest0-21swapper/508:25:167
26299991211cyclictest15803-21cstates09:50:177
2629399123cyclictest0-21swapper/409:25:206
2628599123cyclictest0-21swapper/308:20:195
2627999126cyclictest0-21swapper/208:15:104
2627999125cyclictest0-21swapper/210:35:214
2627999123cyclictest0-21swapper/210:05:004
2627999122cyclictest13385-21python09:00:254
2627999122cyclictest0-21swapper/211:30:234
2627999122cyclictest0-21swapper/210:48:134
2627999122cyclictest0-21swapper/210:48:134
2627999122cyclictest0-21swapper/209:50:014
2627999122cyclictest0-21swapper/208:00:234
2627999122cyclictest0-21swapper/207:45:194
2627999122cyclictest0-21swapper/207:35:014
2627999121cyclictest0-21swapper/212:35:234
2627999121cyclictest0-21swapper/211:08:364
2627999121cyclictest0-21swapper/208:25:274
2627999121cyclictest0-21swapper/207:30:014
2626499125cyclictest0-21swapper/010:15:140
2626499124cyclictest0-21swapper/011:20:520
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional