You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 15:13
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Wed Dec 03, 2025 12:46:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212772122108sleep100-21swapper/1007:05:382
21462210291sleep110-21swapper/1107:08:043
3066350950irq/52-eth0-rx-0-21swapper/507:05:327
3066350930irq/52-eth0-rx-0-21swapper/207:05:224
2161829388sleep30-21swapper/307:09:325
12250900irq/42-ahci0-21swapper/107:05:071
2162728481sleep90-21swapper/907:09:3811
3066450830irq/53-eth0-tx-0-21swapper/407:08:416
2133026653sleep60-21swapper/607:06:238
1848726554sleep00-21swapper/007:05:050
2158426462sleep80-21swapper/807:09:0010
2128926055sleep70-21swapper/707:05:489
21757992625cyclictest15294-21nscd07:15:140
2175799220cyclictest0-21swapper/007:45:250
2175799210cyclictest0-21swapper/011:25:230
2175799200cyclictest0-21swapper/011:20:270
2175799200cyclictest0-21swapper/010:50:250
2175799200cyclictest0-21swapper/010:50:250
2175799200cyclictest0-21swapper/007:25:260
2180499195cyclictest0-21swapper/907:25:0011
21757991917cyclictest0-21swapper/009:08:160
2175799190cyclictest0-21swapper/012:00:250
2175799190cyclictest0-21swapper/011:35:250
2175799190cyclictest0-21swapper/011:15:270
2175799190cyclictest0-21swapper/009:40:260
21805991817cyclictest0-21swapper/1012:22:252
21805991817cyclictest0-21swapper/1010:50:182
21805991817cyclictest0-21swapper/1010:50:182
21805991817cyclictest0-21swapper/1007:10:232
2180599181cyclictest831ksoftirqd/1012:15:292
2180599181cyclictest831ksoftirqd/1011:20:002
2180599181cyclictest831ksoftirqd/1010:35:142
2180599181cyclictest831ksoftirqd/1010:35:142
2180599181cyclictest12250irq/42-ahci09:08:542
2180499181cyclictest12250irq/42-ahci07:40:2011
2180499181cyclictest0-21swapper/909:15:0211
2180499181cyclictest0-21swapper/908:33:2411
21803991816cyclictest7113-21diskstats08:15:1310
21803991816cyclictest6990-21snmp_rack3slot907:30:1610
21803991816cyclictest4667-21perl09:40:1610
21803991816cyclictest31124-21iostat_ios11:00:2010
21803991816cyclictest29090-21snmp_rack3slot911:40:4910
21803991816cyclictest28990-21memory07:15:2610
21803991816cyclictest28750-21snmp_rack3slot910:56:3610
21803991816cyclictest26732-21snmp_rack3slot910:10:1610
21803991816cyclictest24627-21iostat12:20:2010
21803991816cyclictest2117-21snmp_rack3slot908:50:4910
21803991816cyclictest18391-21perl08:30:1510
21803991816cyclictest15312-21snmp_rack3slot910:40:1410
2176699183cyclictest0-21swapper/108:35:171
2176699182cyclictest0-21swapper/111:30:011
21757991816cyclictest20653-21chrt09:18:070
2175799180cyclictest0-21swapper/012:10:270
2175799180cyclictest0-21swapper/011:45:260
2175799180cyclictest0-21swapper/010:15:240
21806991716cyclictest280672sleep1109:29:443
21805991716cyclictest9812-21smartctl12:00:152
21805991716cyclictest8880-21python09:00:272
21805991716cyclictest831ksoftirqd/1011:50:132
21805991716cyclictest831ksoftirqd/1009:25:142
21805991716cyclictest831ksoftirqd/1008:05:152
21805991716cyclictest7789-21expr09:45:132
21805991716cyclictest5227-21cat08:10:282
21805991716cyclictest26602-21latency_hist12:25:012
21805991716cyclictest20235-21grep10:45:192
21805991716cyclictest18014-1kworker/10:1H09:10:022
21805991716cyclictest16647-21sensors09:55:262
2180599170cyclictest90452sleep1008:15:342
2180599170cyclictest831ksoftirqd/1011:05:122
2180599170cyclictest831ksoftirqd/1010:00:252
2180599170cyclictest831ksoftirqd/1009:20:202
2180599170cyclictest831ksoftirqd/1008:55:132
2180599170cyclictest831ksoftirqd/1008:35:122
2180599170cyclictest76902chrt11:56:352
2180599170cyclictest73902chrt11:12:182
2180599170cyclictest6482sleep1012:30:512
2180599170cyclictest63362sleep1009:40:442
2180599170cyclictest49372sleep1007:26:052
2180599170cyclictest3252sleep1011:46:452
2180599170cyclictest323402sleep1011:00:532
2180599170cyclictest319452sleep1010:16:132
2180599170cyclictest314912chrt09:31:422
2180599170cyclictest3066350irq/52-eth0-rx-10:25:132
2180599170cyclictest3066350irq/52-eth0-rx-08:45:142
2180599170cyclictest3066350irq/52-eth0-rx-08:40:192
2180599170cyclictest3066350irq/52-eth0-rx-08:40:192
2180599170cyclictest3066350irq/52-eth0-rx-08:26:132
2180599170cyclictest3066350irq/52-eth0-rx-08:26:122
2180599170cyclictest3066350irq/52-eth0-rx-07:35:152
2180599170cyclictest299802sleep1007:15:392
2180599170cyclictest290652sleep1011:40:332
2180599170cyclictest288122chrt10:57:272
2180599170cyclictest285372sleep1010:14:232
2180599170cyclictest268652sleep1007:57:072
2180599170cyclictest25572sleep1010:20:252
2180599170cyclictest254332sleep1011:35:322
2180599170cyclictest246032sleep1010:06:112
2180599170cyclictest23962sleep1012:35:152
2180599170cyclictest22152sleep1008:52:032
2180599170cyclictest218762sleep1011:31:322
2180599170cyclictest206632sleep1009:18:172
2180599170cyclictest177572sleep1012:10:242
2180599170cyclictest158452sleep1007:40:532
2180599170cyclictest157232sleep1010:40:142
2180599170cyclictest147012sleep1011:22:552
2180599170cyclictest13450irq/18-ehci_hcd07:50:252
2180599170cyclictest127872sleep1008:21:592
2180599170cyclictest12042chrt07:21:152
2180599170cyclictest11592sleep1009:35:172
2180599170cyclictest108062sleep1009:50:092
2180599170cyclictest106842sleep1010:33:382
21803991715cyclictest8564-21snmp_rack3slot910:30:1510
21803991715cyclictest5042-21ntp_states08:55:2410
21803991715cyclictest4080-21snmp_rack3slot911:51:4810
21803991715cyclictest405-21snmp_rack3slot909:35:1510
21803991715cyclictest31475-21perl12:30:1510
21803991715cyclictest31368-21snmp_rack3slot907:20:1210
21803991715cyclictest30537-21snmp_rack3slot908:02:0410
21803991715cyclictest30177-21snmp_easybox.os08:45:2410
21803991715cyclictest29726-21perl09:30:1510
21803991715cyclictest27924-21perl12:25:1710
21803991715cyclictest25821-21snmp_rack3slot908:40:1510
21803991715cyclictest25821-21snmp_rack3slot908:40:1510
21803991715cyclictest24238-21iostat11:35:1910
21803991715cyclictest23874-21irqstats07:10:2110
21803991715cyclictest22300-21snmp_rack3slot909:20:1410
21803991715cyclictest21260-21snmp_rack3slot908:35:1210
21803991715cyclictest2011-21munin-run08:10:0110
21803991715cyclictest19722-21perl10:45:1510
21803991715cyclictest18543-21snmp_rack3slot909:15:1410
21803991715cyclictest16580-21perl11:25:1410
21803991715cyclictest14606-21snmp_rack3slot911:21:3910
21803991715cyclictest14577-21snmp_rack3slot908:25:1510
21803991715cyclictest14577-21snmp_rack3slot908:25:1510
21803991715cyclictest10398-21snmp_easybox.os09:05:1110
21792991715cyclictest24805-21qemu-kvm10:08:348
21791991716cyclictest4321-21python07:25:267
21766991715cyclictest6679-21python11:10:261
21806991614cyclictest5214-21chrt07:29:533
21806991614cyclictest300102chrt07:16:043
21806991614cyclictest169652sleep1109:12:303
21805991615cyclictest0-21swapper/1008:31:542
2180499165cyclictest0-21swapper/910:10:0111
21804991615cyclictest0-21swapper/912:35:1511
21804991615cyclictest0-21swapper/907:11:3811
21803991615cyclictest26808-21snmp_rack3slot907:56:2510
21803991615cyclictest20623-21perl12:15:1510
21803991614cyclictest7385-21snmp_rack3slot911:12:1210
21803991614cyclictest4431-21snmp_rack3slot912:36:2510
21803991614cyclictest3809-21perl08:10:1510
21803991614cyclictest30189-21perl10:15:1410
21803991614cyclictest2969-21df_inode07:25:1510
21803991614cyclictest15586-21perl09:55:1710
21803991614cyclictest15299-21snmp_rack3slot909:10:1610
21803991614cyclictest13632-21snmp_rack3slot909:50:5910
21803991614cyclictest10938-21snmp_rack3slot911:16:1510
21791991615cyclictest1860-21qemu-kvm08:08:297
2177399163cyclictest0-21swapper/210:20:164
2177399163cyclictest0-21swapper/208:30:174
2177399163cyclictest0-21swapper/207:50:174
2177399162cyclictest0-21swapper/207:15:214
21766991615cyclictest3066350irq/52-eth0-rx-12:35:501
21757991614cyclictest2691-21runrttasks07:23:150
21757991614cyclictest25186-21chrt07:12:290
2175799160cyclictest0-21swapper/011:00:260
2180699150cyclictest0-21swapper/1108:17:113
21805991514cyclictest17576-21python11:25:252
2180499154cyclictest0-21swapper/908:10:0111
21804991513cyclictest21386-21sshd10:45:4011
21804991513cyclictest11695-21memory08:20:2311
21803991514cyclictest5007-21snmp_rack3slot911:55:1310
21803991514cyclictest32080-21snmp_rack3slot911:45:2410
21803991513cyclictest7839-21perl09:00:1410
21803991513cyclictest7628-21snmp_rack3slot909:45:1310
21803991513cyclictest26170-21perl09:25:1510
21803991513cyclictest22492-21snmp_rack3slot907:50:2310
21803991513cyclictest20596-21iostat_ios11:30:2010
21803991513cyclictest11376-21snmp_rack3slot912:01:5010
21785991513cyclictest23532-21sensors09:20:256
2177999150cyclictest0-21swapper/307:47:365
2177399153cyclictest0-21swapper/208:00:194
2177399150cyclictest0-21swapper/211:05:184
2176699152cyclictest0-21swapper/110:00:001
2175799150cyclictest0-21swapper/010:25:120
2175799150cyclictest0-21swapper/009:47:280
21806991412cyclictest25329-21chrt07:14:273
21806991412cyclictest20508-21chrt09:16:063
2180599144cyclictest0-21swapper/1007:45:012
21805991413cyclictest0-21swapper/1012:06:112
2180499144cyclictest0-21swapper/912:09:5911
2180499144cyclictest0-21swapper/907:30:0111
2180499143cyclictest0-21swapper/908:40:0111
2180499143cyclictest0-21swapper/908:40:0111
21803991412cyclictest17352-21iostat12:10:2010
21803991412cyclictest1607-21df_inode11:05:1410
21803991412cyclictest1543-21perl10:20:1410
21803991412cyclictest13510-21sensors_fan10:35:2610
21803991412cyclictest13510-21sensors_fan10:35:2610
2179699145cyclictest0-21swapper/708:40:189
2179699145cyclictest0-21swapper/708:40:179
2179299147cyclictest20804-21munin-run09:20:018
2179199140cyclictest24805-21qemu-kvm10:45:157
21785991413cyclictest10653-21sensors12:00:256
21785991412cyclictest30816-21sensors09:30:256
2177399142cyclictest0-21swapper/207:35:214
2177399141cyclictest0-21swapper/211:55:184
2175799144cyclictest0-21swapper/009:28:410
2175799143cyclictest0-21swapper/010:08:580
2175799141cyclictest0-21swapper/009:10:130
2175799141cyclictest0-21swapper/008:10:170
2175799141cyclictest0-21swapper/008:05:000
2175799140cyclictest0-21swapper/012:35:100
2175799140cyclictest0-21swapper/008:25:230
2175799140cyclictest0-21swapper/008:25:230
2175799140cyclictest0-21swapper/007:55:010
2175799140cyclictest0-21swapper/007:30:560
2180699133cyclictest881rcuc/1110:55:143
2180699133cyclictest881rcuc/1108:00:013
2180699133cyclictest0-21swapper/1112:31:573
2180699133cyclictest0-21swapper/1112:11:563
2180699133cyclictest0-21swapper/1109:04:403
2180699133cyclictest0-21swapper/1108:58:553
2180699133cyclictest0-21swapper/1107:43:063
21806991311cyclictest24282-21chrt09:22:433
2180499134cyclictest0-21swapper/911:25:0111
2180499133cyclictest0-21swapper/912:25:0111
2180499133cyclictest0-21swapper/911:25:0111
2180499133cyclictest0-21swapper/910:29:2411
2180499133cyclictest0-21swapper/910:17:1111
2180499133cyclictest0-21swapper/909:08:2411
2180499133cyclictest0-21swapper/908:26:2511
2180499133cyclictest0-21swapper/908:26:2411
2180499133cyclictest0-21swapper/907:54:0411
2180499133cyclictest0-21swapper/907:35:4011
21804991312cyclictest0-21swapper/912:10:1511
21804991312cyclictest0-21swapper/908:45:1411
21803991311cyclictest23121-21hddtemp_smartct10:05:1710
21803991311cyclictest14212-21snmp_rack3slot912:05:2810
21803991311cyclictest10991-21iostat_ios07:35:2110
2179699133cyclictest0-21swapper/710:40:029
2179699133cyclictest0-21swapper/709:45:089
21792991311cyclictest24805-21qemu-kvm10:46:098
21791991312cyclictest23979-21python10:05:277
21785991312cyclictest3391-21sensors11:50:246
2177999133cyclictest0-21swapper/309:30:175
2177999133cyclictest0-21swapper/307:40:185
2176699131cyclictest0-21swapper/112:10:191
2176699131cyclictest0-21swapper/108:20:011
2175799135cyclictest0-21swapper/010:30:110
2175799133cyclictest736-21munin-node11:05:130
2175799133cyclictest0-21swapper/012:20:030
2175799131cyclictest0-21swapper/011:41:030
2175799131cyclictest0-21swapper/011:10:150
2175799131cyclictest0-21swapper/009:35:160
2175799131cyclictest0-21swapper/008:15:150
2175799131cyclictest0-21swapper/007:35:170
2175799130cyclictest20316-21munin-node07:50:200
2175799130cyclictest14451-21munin-node09:55:250
2175799130cyclictest0-21swapper/012:06:010
2175799130cyclictest0-21swapper/011:55:460
2175799130cyclictest0-21swapper/010:56:110
2175799130cyclictest0-21swapper/010:41:240
2175799130cyclictest0-21swapper/010:20:170
2175799130cyclictest0-21swapper/010:13:410
2175799130cyclictest0-21swapper/010:01:380
2175799130cyclictest0-21swapper/009:52:100
2175799130cyclictest0-21swapper/009:35:000
2175799130cyclictest0-21swapper/008:51:080
2175799130cyclictest0-21swapper/008:40:060
2175799130cyclictest0-21swapper/008:40:050
2175799130cyclictest0-21swapper/008:35:110
2175799130cyclictest0-21swapper/008:34:480
2175799130cyclictest0-21swapper/008:00:300
2180699124cyclictest0-21swapper/1111:16:413
2180699122cyclictest0-21swapper/1112:04:423
2180699122cyclictest0-21swapper/1111:36:103
2180699122cyclictest0-21swapper/1109:06:183
2180699122cyclictest0-21swapper/1108:07:553
21806991211cyclictest2027-21fschecks_count11:50:133
21806991210cyclictest881rcuc/1110:10:203
21806991210cyclictest0-21swapper/1108:30:253
21805991211cyclictest0-21swapper/1008:02:262
2180499123cyclictest0-21swapper/910:30:0111
2180499123cyclictest0-21swapper/910:10:0111
2180499123cyclictest0-21swapper/909:47:2511
2180499123cyclictest0-21swapper/908:15:0111
2180499123cyclictest0-21swapper/907:15:2411
2180499122cyclictest0-21swapper/912:31:1311
2180499122cyclictest0-21swapper/912:00:3011
2180499122cyclictest0-21swapper/911:33:4011
2180499122cyclictest0-21swapper/909:33:4011
2180499122cyclictest0-21swapper/909:23:3911
2180499122cyclictest0-21swapper/908:14:5311
2180499122cyclictest0-21swapper/907:23:5511
21804991211cyclictest0-21swapper/908:00:3211
21803991210cyclictest23842-21iostat_ios10:50:1810
21803991210cyclictest23842-21iostat_ios10:50:1810
21803991210cyclictest14830-21memory07:40:2110
2179699126cyclictest0-21swapper/710:24:089
2179699124cyclictest0-21swapper/711:15:019
2179699122cyclictest0-21swapper/710:14:199
2179699122cyclictest0-21swapper/708:39:069
21796991210cyclictest0-21swapper/712:00:219
2179699120cyclictest0-21swapper/712:39:559
2179299125cyclictest13932-21munin-run09:55:008
2179299123cyclictest0-21swapper/612:10:008
2179299122cyclictest0-21swapper/610:29:088
2179299120cyclictest0-21swapper/607:19:038
21791991211cyclictest7154-21idleruntime07:30:187
21791991211cyclictest13362-21qemu-kvm09:43:197
21791991211cyclictest13362-21qemu-kvm09:08:257
2177999129cyclictest0-21swapper/310:26:155
2177399122cyclictest0-21swapper/209:20:014
2176699124cyclictest0-21swapper/111:20:191
2176699122cyclictest0-21swapper/107:15:221
21766991211cyclictest21600-21python12:15:261
2175799126cyclictest21755-21cyclictest11:50:240
2175799122cyclictest0-21swapper/008:21:150
2175799121cyclictest0-21swapper/012:25:260
2175799121cyclictest0-21swapper/012:15:150
2175799121cyclictest0-21swapper/010:45:140
2175799121cyclictest0-21swapper/010:35:140
2175799121cyclictest0-21swapper/010:35:140
2175799121cyclictest0-21swapper/008:55:130
2175799121cyclictest0-21swapper/007:40:110
2175799120cyclictest0-21swapper/012:30:280
2175799120cyclictest0-21swapper/011:30:050
2175799120cyclictest0-21swapper/009:20:300
2175799120cyclictest0-21swapper/009:00:040
2175799120cyclictest0-21swapper/008:45:130
761110ksoftirqd/90-21swapper/911:50:1911
3066450110irq/53-eth0-tx-0-21swapper/409:45:126
2180699119cyclictest881rcuc/1111:27:243
2180699119cyclictest0-21swapper/1107:32:173
2180699114cyclictest0-21swapper/1109:42:113
2180699113cyclictest0-21swapper/1112:25:593
2180699113cyclictest0-21swapper/1111:20:563
2180699113cyclictest0-21swapper/1110:15:343
2180699113cyclictest0-21swapper/1110:00:403
2180699113cyclictest0-21swapper/1109:45:303
2180699113cyclictest0-21swapper/1108:47:333
2180699113cyclictest0-21swapper/1108:20:393
2180699113cyclictest0-21swapper/1108:10:063
2180699113cyclictest0-21swapper/1108:01:583
2180699112cyclictest901ksoftirqd/1109:50:293
2180699112cyclictest0-21swapper/1112:08:413
2180699111cyclictest0-21swapper/1111:59:423
2180699110cyclictest0-21swapper/1111:45:243
2180699110cyclictest0-21swapper/1110:05:313
21805991110cyclictest831ksoftirqd/1007:30:302
2180499119cyclictest32603-21diskstats08:50:1411
2180499113cyclictest0-21swapper/912:16:3411
2180499113cyclictest0-21swapper/911:35:5511
2180499113cyclictest0-21swapper/910:52:3811
2180499113cyclictest0-21swapper/910:52:3811
2180499113cyclictest0-21swapper/910:40:2511
2180499113cyclictest0-21swapper/909:40:2611
2180499113cyclictest0-21swapper/909:39:3711
21804991110cyclictest0-21swapper/911:55:1911
21804991110cyclictest0-21swapper/908:35:2311
2180399119cyclictest21027-21snmp_rack3slot910:02:0210
2180399113cyclictest0-21swapper/810:28:0810
2180399113cyclictest0-21swapper/808:20:3710
2179699119cyclictest0-21swapper/712:30:289
2179699119cyclictest0-21swapper/711:27:019
2179699119cyclictest0-21swapper/711:00:559
2179699119cyclictest0-21swapper/709:36:219
2179699115cyclictest0-21swapper/712:17:129
2179699115cyclictest0-21swapper/707:27:549
2179699113cyclictest4539-21snmp_easybox.os11:10:119
2179699113cyclictest0-21swapper/711:42:109
2179699113cyclictest0-21swapper/710:50:299
2179699113cyclictest0-21swapper/710:50:299
2179699113cyclictest0-21swapper/710:01:499
2179699113cyclictest0-21swapper/709:32:389
2179699113cyclictest0-21swapper/709:18:559
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional