You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sat Jul 12, 2025 00:46:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28790210693sleep70-21swapper/719:06:019
128501040irq/42-ahci0-21swapper/119:08:481
28839210391sleep60-21swapper/619:06:368
2912029893sleep90-21swapper/919:09:3911
12850950irq/42-ahci0-21swapper/219:08:254
2014050930irq/53-eth0-rx-0-21swapper/519:05:247
2014050930irq/53-eth0-rx-0-21swapper/419:05:506
12850880irq/42-ahci0-21swapper/819:08:3710
2758228782sleep30-21swapper/319:05:215
2887626455sleep110-21swapper/1119:07:053
125250640irq/18-parport00-21swapper/1019:05:232
2878325955sleep00-21swapper/019:05:540
2928999310cyclictest0-21swapper/1120:20:053
2928999310cyclictest0-21swapper/1120:19:543
29246993011cyclictest3634-1kworker/0:0H22:30:110
29289992214cyclictest0-21swapper/1121:14:273
2928999210cyclictest0-21swapper/1121:21:213
2924699200cyclictest0-21swapper/022:11:500
2928899192cyclictest831ksoftirqd/1022:40:302
2928899192cyclictest831ksoftirqd/1000:07:232
29289991817cyclictest2014150irq/54-eth0-tx-21:15:143
29288991817cyclictest831ksoftirqd/1019:15:152
29288991817cyclictest811rcuc/1021:57:582
29288991816cyclictest95072sleep1020:09:112
2928899181cyclictest831ksoftirqd/1022:00:162
2928899181cyclictest831ksoftirqd/1020:55:132
2928899181cyclictest831ksoftirqd/1020:55:132
2928899180cyclictest67912sleep1021:34:062
2928799181cyclictest10821-21taskset22:22:4011
2928799181cyclictest0-21swapper/922:45:1411
2928799181cyclictest0-21swapper/920:11:1211
29284991816cyclictest20848-21qemu-kvm21:09:238
29246991816cyclictest2423-21snmpd20:19:200
2928999170cyclictest0-21swapper/1121:08:173
29288991716cyclictest13165-21perl23:10:172
29288991715cyclictest20485-21snmp_easybox.os21:50:282
2928899170cyclictest99622sleep1000:35:152
2928899170cyclictest8812sleep1000:20:502
2928899170cyclictest831ksoftirqd/1023:30:172
2928899170cyclictest831ksoftirqd/1021:25:162
2928899170cyclictest831ksoftirqd/1020:52:172
2928899170cyclictest831ksoftirqd/1020:52:162
2928899170cyclictest831ksoftirqd/1019:20:112
2928899170cyclictest811rcuc/1022:20:012
2928899170cyclictest80072sleep1023:47:582
2928899170cyclictest80072sleep1023:47:582
2928899170cyclictest7502sleep1023:39:342
2928899170cyclictest7502sleep1023:39:342
2928899170cyclictest72372sleep1022:18:232
2928899170cyclictest61232chrt20:47:032
2928899170cyclictest44242sleep1023:43:332
2928899170cyclictest35112sleep1022:55:292
2928899170cyclictest321572sleep1022:07:112
2928899170cyclictest305902sleep1019:50:322
2928899170cyclictest280292sleep1021:16:412
2928899170cyclictest206872sleep1021:06:402
2928899170cyclictest2014050irq/53-eth0-rx-21:40:252
2928899170cyclictest193122sleep1020:20:242
2928899170cyclictest188492sleep1000:01:362
2928899170cyclictest188492sleep1000:01:352
2928899170cyclictest174782sleep1021:46:082
2928899170cyclictest170992sleep1021:02:172
2928899170cyclictest170992sleep1021:02:172
2928899170cyclictest143292chrt22:25:482
2928899170cyclictest12850irq/42-ahci19:38:362
2928899170cyclictest12850irq/42-ahci19:10:212
2928899170cyclictest124312sleep1019:26:052
2928899170cyclictest103782sleep1021:38:322
2928799175cyclictest0-21swapper/923:40:1611
29287991715cyclictest27691-21chrt20:33:5911
29287991715cyclictest24851-21ntpdc23:25:2111
29287991715cyclictest22125-21chrt23:22:0611
29287991715cyclictest17631-21taskset21:48:0511
2928799170cyclictest0-21swapper/920:21:3411
29284991715cyclictest7662-21python00:30:298
29284991715cyclictest17791-21python23:15:258
29280991716cyclictest9429-21qemu-kvm20:08:267
31160ksoftirqd/00-21swapper/023:50:260
2928999160cyclictest0-21swapper/1122:23:163
29288991615cyclictest831ksoftirqd/1022:50:262
29288991615cyclictest831ksoftirqd/1020:40:152
29288991615cyclictest21015-21python22:35:272
29288991615cyclictest0-21swapper/1023:05:302
29288991614cyclictest21075-21snmp_easybox.os20:25:122
29287991615cyclictest0-21swapper/922:10:1311
29287991615cyclictest0-21swapper/920:01:2711
29287991615cyclictest0-21swapper/919:35:1511
29287991615cyclictest0-21swapper/919:30:2411
29287991614cyclictest880-21chrt00:20:4911
29287991614cyclictest25523-21chrt22:44:5011
2925999161cyclictest0-21swapper/223:20:174
2925999161cyclictest0-21swapper/223:05:304
2924899161cyclictest0-21swapper/122:50:151
29246991615cyclictest2415-21nscd20:26:490
29246991614cyclictest20433-21chrt20:24:140
2924699160cyclictest0-21swapper/019:10:370
8799150watchdog/110-21swapper/1119:25:283
29289991514cyclictest12850irq/42-ahci19:10:203
29288991514cyclictest831ksoftirqd/1020:00:282
29288991514cyclictest0-21swapper/1023:00:132
29288991514cyclictest0-21swapper/1021:10:142
29288991514cyclictest0-21swapper/1020:10:142
29287991514cyclictest32599-21timerandwakeup23:35:3211
29287991514cyclictest32599-21timerandwakeup23:35:3211
29287991514cyclictest26193-1kworker/9:2H21:40:4811
29287991514cyclictest0-21swapper/900:06:4711
29287991513cyclictest14522-21taskset22:28:2711
2928799151cyclictest0-21swapper/919:10:1911
2928699150cyclictest0-21swapper/819:10:1410
2928599153cyclictest0-21swapper/722:05:019
2928599150cyclictest0-21swapper/722:12:389
2927399150cyclictest0-21swapper/421:45:126
2927399150cyclictest0-21swapper/420:15:206
2926699150cyclictest0-21swapper/321:10:245
2926699150cyclictest0-21swapper/320:00:225
2926699150cyclictest0-21swapper/300:30:325
29259991513cyclictest2423-21snmpd19:54:284
29246991514cyclictest143622chrt22:26:150
2924699150cyclictest0-21swapper/022:15:210
2924699150cyclictest0-21swapper/021:20:260
31140ksoftirqd/00-21swapper/023:55:270
31140ksoftirqd/00-21swapper/023:55:260
2928999144cyclictest0-21swapper/1123:55:023
2928999144cyclictest0-21swapper/1123:55:023
2928999143cyclictest0-21swapper/1123:45:013
29288991413cyclictest831ksoftirqd/1020:15:132
29288991413cyclictest811rcuc/1019:55:162
2928799144cyclictest0-21swapper/920:55:0111
2928799144cyclictest0-21swapper/920:55:0111
2928799143cyclictest0-21swapper/919:15:0111
29287991413cyclictest0-21swapper/921:30:2211
29287991413cyclictest0-21swapper/921:07:0311
2928699144cyclictest0-21swapper/820:10:0010
2928699143cyclictest0-21swapper/819:45:0110
2928699143cyclictest0-21swapper/800:35:0110
29286991412cyclictest21413-21awk21:55:0010
2928599143cyclictest0-21swapper/723:45:019
2928599143cyclictest0-21swapper/723:45:019
2928599143cyclictest0-21swapper/721:00:019
2928599143cyclictest0-21swapper/721:00:019
2928599143cyclictest0-21swapper/720:55:029
2928599143cyclictest0-21swapper/720:55:019
2928499142cyclictest0-21swapper/600:01:478
2928499142cyclictest0-21swapper/600:01:478
29284991413cyclictest9075-21python20:50:258
29284991413cyclictest9075-21python20:50:258
29259991413cyclictest2799-21python22:10:284
2924899140cyclictest0-21swapper/120:10:121
3199130watchdog/30-21swapper/322:14:235
31130ksoftirqd/00-21swapper/023:30:240
31130ksoftirqd/00-21swapper/020:55:230
31130ksoftirqd/00-21swapper/020:55:220
31130ksoftirqd/00-21swapper/019:20:220
2928999134cyclictest0-21swapper/1123:55:013
2928999133cyclictest0-21swapper/1120:13:593
29289991311cyclictest19107-21munin-run00:05:013
29289991311cyclictest19107-21munin-run00:05:003
2928999131cyclictest0-21swapper/1123:05:203
29288991312cyclictest27142-21perl22:45:152
29288991311cyclictest670-21snmp_rack3slot922:10:142
2928799134cyclictest0-21swapper/919:55:0111
2928799133cyclictest0-21swapper/920:20:0011
2928799132cyclictest0-21swapper/919:25:0011
29287991312cyclictest26193-1kworker/9:2H00:31:4811
29287991312cyclictest0-21swapper/921:00:1711
29287991312cyclictest0-21swapper/921:00:1711
29287991312cyclictest0-21swapper/920:45:1711
2928699133cyclictest0-21swapper/821:15:0210
2928599133cyclictest0-21swapper/722:57:319
2928599133cyclictest0-21swapper/721:20:029
2928599133cyclictest0-21swapper/721:10:019
2928499137cyclictest9429-21qemu-kvm20:15:008
2928499135cyclictest0-21swapper/600:00:018
2928499135cyclictest0-21swapper/600:00:008
2928499133cyclictest0-21swapper/621:19:168
2928099132cyclictest0-21swapper/519:53:467
2928099131cyclictest0-21swapper/500:09:327
2924899133cyclictest0-21swapper/122:10:181
2924699130cyclictest0-21swapper/019:45:280
31120ksoftirqd/00-21swapper/022:00:270
31120ksoftirqd/00-21swapper/020:40:260
2928999123cyclictest0-21swapper/1122:00:003
2928999123cyclictest0-21swapper/1100:14:023
2928999122cyclictest0-21swapper/1122:59:323
2928999122cyclictest0-21swapper/1100:27:173
2928899123cyclictest0-21swapper/1019:30:012
29288991211cyclictest831ksoftirqd/1019:45:252
2928799124cyclictest0-21swapper/922:05:0011
29287991211cyclictest26193-1kworker/9:2H23:30:2211
29287991211cyclictest26193-1kworker/9:2H21:29:2211
29287991211cyclictest26193-1kworker/9:2H20:52:1311
29287991211cyclictest26193-1kworker/9:2H20:52:1211
29287991211cyclictest0-21swapper/919:20:2511
2928699123cyclictest0-21swapper/821:07:0010
2928699122cyclictest0-21swapper/822:20:0110
2928599124cyclictest0-21swapper/721:30:019
2928599124cyclictest0-21swapper/721:25:019
2928599122cyclictest0-21swapper/720:25:009
2928499124cyclictest0-21swapper/620:09:178
2928499123cyclictest0-21swapper/619:30:008
2928499122cyclictest0-21swapper/619:17:438
29284991210cyclictest3617-21python23:40:238
29284991210cyclictest25051-21python23:25:238
2928099124cyclictest0-21swapper/523:20:197
2928099124cyclictest0-21swapper/500:20:187
2928099122cyclictest0-21swapper/520:20:587
2927399122cyclictest0-21swapper/421:40:026
2925999122cyclictest0-21swapper/223:50:194
2925999122cyclictest0-21swapper/221:05:184
2925999122cyclictest0-21swapper/219:15:204
2924699120cyclictest0-21swapper/021:00:260
2924699120cyclictest0-21swapper/021:00:260
6199110migration/724456-21runrttasks21:12:369
551110ksoftirqd/629007-21qemu-kvm19:25:288
5499110migration/65910-21munin-node23:45:138
5499110migration/65910-21munin-node23:45:128
481110ksoftirqd/50-21swapper/523:50:137
31110ksoftirqd/00-21swapper/023:10:270
31110ksoftirqd/00-21swapper/020:05:250
31110ksoftirqd/00-21swapper/019:25:280
2928999119cyclictest0-21swapper/1120:40:263
2928999113cyclictest0-21swapper/1123:46:023
2928999113cyclictest0-21swapper/1123:46:023
2928999113cyclictest0-21swapper/1123:38:173
2928999113cyclictest0-21swapper/1123:38:173
2928999113cyclictest0-21swapper/1123:20:023
2928999113cyclictest0-21swapper/1123:15:363
2928999113cyclictest0-21swapper/1122:30:163
2928999113cyclictest0-21swapper/1120:47:443
2928999113cyclictest0-21swapper/1100:35:173
2928999113cyclictest0-21swapper/1100:22:033
2928899119cyclictest22703-21snmp_easybox.os19:40:272
2928899113cyclictest0-21swapper/1023:50:002
29288991110cyclictest26622-21munin-node20:30:202
29288991110cyclictest0-21swapper/1022:30:262
29288991110cyclictest0-21swapper/1020:35:492
29288991110cyclictest0-21swapper/1020:35:492
29288991110cyclictest0-21swapper/1000:15:212
2928799113cyclictest0-21swapper/923:09:2511
2928799113cyclictest0-21swapper/921:24:4611
2928799113cyclictest0-21swapper/921:20:0011
2928799112cyclictest0-21swapper/919:45:0111
29287991110cyclictest0-21swapper/922:50:1711
29287991110cyclictest0-21swapper/922:02:4011
29287991110cyclictest0-21swapper/921:50:4111
2928699114cyclictest12850irq/42-ahci20:00:0010
2928699113cyclictest0-21swapper/823:15:5510
2928699113cyclictest0-21swapper/821:03:4910
2928699113cyclictest0-21swapper/821:03:4810
2928699113cyclictest0-21swapper/820:45:2110
2928699112cyclictest0-21swapper/823:27:3210
2928699112cyclictest0-21swapper/822:14:4510
2928699112cyclictest0-21swapper/821:47:3110
2928599113cyclictest0-21swapper/723:18:029
2928599113cyclictest0-21swapper/723:04:179
2928599113cyclictest0-21swapper/721:54:019
2928599113cyclictest0-21swapper/720:13:149
2928599113cyclictest0-21swapper/719:40:009
2928599113cyclictest0-21swapper/700:25:029
2928599112cyclictest0-21swapper/700:10:019
2928599111cyclictest0-21swapper/722:49:599
2928599110cyclictest0-21swapper/723:40:189
2928499113cyclictest29008-21qemu-kvm19:45:168
2928499113cyclictest20848-21qemu-kvm21:45:368
2928499113cyclictest0-21swapper/623:06:328
2928499113cyclictest0-21swapper/622:47:478
2928499113cyclictest0-21swapper/622:23:238
2928499113cyclictest0-21swapper/621:23:458
2928499113cyclictest0-21swapper/621:03:308
2928499113cyclictest0-21swapper/621:03:308
2928499113cyclictest0-21swapper/620:49:298
2928499113cyclictest0-21swapper/620:40:018
2928499113cyclictest0-21swapper/620:40:008
2928499110cyclictest0-21swapper/621:00:018
2928499110cyclictest0-21swapper/621:00:008
2928099115cyclictest9430-21qemu-kvm20:36:277
2928099115cyclictest9430-21qemu-kvm20:36:277
2928099114cyclictest0-21swapper/522:17:377
2928099113cyclictest29008-21qemu-kvm19:28:077
2928099113cyclictest20848-21qemu-kvm21:10:517
2928099113cyclictest0-21swapper/523:05:197
2928099113cyclictest0-21swapper/522:41:017
2928099113cyclictest0-21swapper/521:00:017
2928099113cyclictest0-21swapper/521:00:017
2928099113cyclictest0-21swapper/519:35:207
2928099112cyclictest0-21swapper/522:55:187
2928099112cyclictest0-21swapper/522:45:207
2928099112cyclictest0-21swapper/521:52:447
2928099111cyclictest0-21swapper/522:29:567
2927399113cyclictest0-21swapper/400:20:166
2927399113cyclictest0-21swapper/400:00:016
2927399113cyclictest0-21swapper/400:00:016
2927399110cyclictest0-21swapper/421:55:166
2926699113cyclictest0-21swapper/321:25:195
2925999118cyclictest0-21swapper/223:40:264
2925999113cyclictest0-21swapper/223:25:164
2925999113cyclictest0-21swapper/220:25:174
2925999113cyclictest0-21swapper/220:20:014
2925999113cyclictest0-21swapper/200:10:194
2925999110cyclictest5793-21snmp_easybox.os21:30:244
2925999110cyclictest0-21swapper/223:10:314
2925999110cyclictest0-21swapper/221:10:124
2924899113cyclictest0-21swapper/123:10:211
2924899113cyclictest0-21swapper/122:25:151
2924899113cyclictest0-21swapper/121:55:201
2924899111cyclictest0-21swapper/123:40:161
2924699119cyclictest0-21swapper/000:25:250
2924699112cyclictest0-21swapper/021:10:230
29246991110cyclictest109672chrt22:24:380
2924699110cyclictest0-21swapper/021:08:170
2924699110cyclictest0-21swapper/019:15:000
2924699110cyclictest0-21swapper/000:35:130
201110ksoftirqd/10-21swapper/119:50:011
901100ksoftirqd/110-21swapper/1119:50:013
901100ksoftirqd/110-21swapper/1119:15:283
8299100migration/102739-21fschecks_count00:25:162
761100ksoftirqd/90-21swapper/900:00:2411
761100ksoftirqd/90-21swapper/900:00:2411
691100ksoftirqd/80-21swapper/823:50:1310
691100ksoftirqd/80-21swapper/822:50:3010
691100ksoftirqd/80-21swapper/821:41:3210
691100ksoftirqd/80-21swapper/820:00:1310
691100ksoftirqd/80-21swapper/819:50:0110
6899100migration/816772-21runrttasks20:19:0110
621100ksoftirqd/70-21swapper/719:50:019
621100ksoftirqd/70-21swapper/700:21:369
6199100migration/729667-21latency_hist19:10:019
5999100watchdog/70-21swapper/720:05:249
5499100migration/630882-21cpuspeed_turbos00:20:128
5499100migration/618598-21latency_hist22:35:018
5299100watchdog/60-21swapper/621:25:448
5299100watchdog/60-21swapper/621:10:198
481100ksoftirqd/50-21swapper/522:50:317
481100ksoftirqd/50-21swapper/522:10:287
481100ksoftirqd/50-21swapper/519:15:287
481100ksoftirqd/50-21swapper/500:00:247
481100ksoftirqd/50-21swapper/500:00:247
4599100watchdog/50-21swapper/519:44:437
411100ksoftirqd/40-21swapper/423:50:136
3899100watchdog/40-21swapper/423:44:396
341100ksoftirqd/30-21swapper/323:50:135
341100ksoftirqd/30-21swapper/323:25:175
341100ksoftirqd/30-21swapper/322:50:305
31100ksoftirqd/00-21swapper/023:25:170
31100ksoftirqd/00-21swapper/021:41:310
31100ksoftirqd/00-21swapper/021:17:430
31100ksoftirqd/00-21swapper/019:50:240
31100ksoftirqd/00-21swapper/019:40:200
2928999104cyclictest0-21swapper/1123:25:013
2928999104cyclictest0-21swapper/1119:36:293
2928999103cyclictest0-21swapper/1122:08:213
2928999103cyclictest0-21swapper/1121:39:503
2928999102cyclictest0-21swapper/1123:30:013
2928999102cyclictest0-21swapper/1123:11:173
2928999102cyclictest0-21swapper/1123:00:323
2928999102cyclictest0-21swapper/1122:35:323
2928999102cyclictest0-21swapper/1120:05:443
2928999102cyclictest0-21swapper/1100:18:313
2928999101cyclictest0-21swapper/1119:47:183
2928999100cyclictest0-21swapper/1122:45:003
2928899109cyclictest831ksoftirqd/1000:30:272
2928899109cyclictest16975-21munin-node23:15:172
2928899109cyclictest0-21swapper/1023:55:312
2928899109cyclictest0-21swapper/1023:55:312
2928899109cyclictest0-21swapper/1023:28:072
2928899109cyclictest0-21swapper/1000:10:062
2928799103cyclictest0-21swapper/922:15:1611
2928799103cyclictest0-21swapper/921:35:0111
2928799103cyclictest0-21swapper/900:29:5811
2928799102cyclictest0-21swapper/923:57:5711
2928799102cyclictest0-21swapper/923:57:5711
2928799102cyclictest0-21swapper/923:15:2111
2928799101cyclictest0-21swapper/922:55:0111
2928699103cyclictest0-21swapper/822:40:4710
2928699103cyclictest0-21swapper/820:40:1910
2928699103cyclictest0-21swapper/820:39:1910
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional