You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 12:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Thu Sep 18, 2025 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
303452205175,12sleep10-21swapper/119:09:401
301282193175,13sleep00-21swapper/019:06:510
300952193176,12sleep30-21swapper/319:06:283
281462184166,13sleep20-21swapper/219:05:032
188082460,0sleep10-21swapper/123:34:131
269032450,0sleep10-21swapper/122:49:181
36452380,0sleep10-21swapper/121:47:491
292992110,0sleep10-21swapper/100:11:291
227932110,0sleep00-21swapper/022:14:570
305599982,3cyclictest1992-21munin-run21:15:010
305599980,4cyclictest25241-21munin-run22:00:010
305599971,3cyclictest32694-21perf23:25:000
305599970,4cyclictest12361-21cron00:20:000
11592270,0sleep20-21swapper/223:46:192
305639962,3cyclictest0-21swapper/122:20:161
305599961,0cyclictest0-21swapper/021:28:470
305769955,0cyclictest12672-21kworker/3:123:36:153
305769955,0cyclictest12672-21kworker/3:100:32:103
305769954,1cyclictest5552-21munin-plugin-st22:55:013
305769950,5cyclictest31685-21ssh23:56:243
305769950,5cyclictest0-21swapper/323:40:493
305769950,5cyclictest0-21swapper/323:30:103
305769950,5cyclictest0-21swapper/322:25:123
305769950,5cyclictest0-21swapper/321:19:333
305769950,4cyclictest24118-21apt-get21:05:013
305769950,0cyclictest0-21swapper/321:44:173
305719955,0cyclictest0-21swapper/223:02:232
305719955,0cyclictest0-21swapper/221:22:002
305719955,0cyclictest0-21swapper/200:13:232
305719950,5cyclictest0-21swapper/222:15:122
305639952,2cyclictest0-21swapper/121:40:121
305639952,2cyclictest0-21swapper/119:20:121
305639952,2cyclictest0-21swapper/119:20:121
305639951,3cyclictest0-21swapper/119:40:121
305639951,2cyclictest0-21swapper/123:55:131
305639951,2cyclictest0-21swapper/122:15:121
305639950,5cyclictest0-21swapper/123:49:011
305639950,5cyclictest0-21swapper/122:28:251
305639950,5cyclictest0-21swapper/121:25:161
305599950,5cyclictest0-21swapper/022:43:070
305599950,5cyclictest0-21swapper/021:20:110
305599950,4cyclictest592-21munin-node19:15:010
305599950,4cyclictest1393-21perl00:30:150
305599950,1cyclictest1406-21munin-run20:20:000
2356250,0sleep10-21swapper/100:30:231
13338250,0sleep10-21swapper/100:20:131
305769944,0cyclictest12672-21kworker/3:122:40:153
305769944,0cyclictest12672-21kworker/3:121:55:273
305769944,0cyclictest12672-21kworker/3:100:09:483
305769944,0cyclictest0-21swapper/322:18:023
305769944,0cyclictest0-21swapper/320:55:223
305769944,0cyclictest0-21swapper/319:18:383
305769943,1cyclictest16425-21grep23:00:143
305769943,1cyclictest14391-21df_inode19:40:143
305769942,1cyclictest0-21swapper/321:25:123
305769941,3cyclictest7887-21ssh23:45:093
305769941,3cyclictest6516-21ssh22:06:063
305769940,4cyclictest6107-21apt-get22:55:143
305769940,4cyclictest5480-21ssh22:38:263
305769940,4cyclictest5480-21ssh22:38:253
305769940,4cyclictest23097-21cut00:25:123
305769940,4cyclictest15255-21ssh21:20:443
305769940,4cyclictest0-21swapper/323:25:133
305769940,4cyclictest0-21swapper/323:09:413
305769940,4cyclictest0-21swapper/322:30:133
305769940,4cyclictest0-21swapper/322:11:123
305769940,4cyclictest0-21swapper/321:51:163
305769940,4cyclictest0-21swapper/321:45:103
305769940,4cyclictest0-21swapper/319:25:093
305769940,4cyclictest0-21swapper/300:20:013
305769940,3cyclictest670-21ssh22:20:023
305769940,3cyclictest31072-21ssh21:12:573
305769940,3cyclictest28463-21ssh22:00:463
305769940,3cyclictest27331-21ssh23:21:373
305769940,3cyclictest27066-21ssh23:54:463
305769940,3cyclictest24171-21apt-get20:00:093
305769940,3cyclictest20051-21ntp_states22:45:173
305769940,3cyclictest18782-21latency_hist19:50:013
305769940,3cyclictest17040-21ssh23:16:243
305769940,3cyclictest12089-21ssh00:35:183
305769940,3cyclictest11524-21sed20:40:003
305769940,3cyclictest0-21swapper/323:10:043
305769940,1cyclictest25764-21grep00:10:153
305769940,0cyclictest0-21swapper/321:39:033
305719944,0cyclictest0-21swapper/222:56:252
305719944,0cyclictest0-21swapper/221:53:252
305719943,1cyclictest11095-21tail21:35:182
305719942,1cyclictest16548-21ssh22:44:072
305719942,1cyclictest15148-21ssh00:05:012
305719942,1cyclictest0-21swapper/220:05:122
305719940,4cyclictest28565-21apt-get22:50:122
305719940,4cyclictest13317-21ssh23:15:112
305719940,4cyclictest0-21swapper/223:53:592
305719940,4cyclictest0-21swapper/223:35:552
305719940,4cyclictest0-21swapper/223:25:362
305719940,4cyclictest0-21swapper/223:09:462
305719940,4cyclictest0-21swapper/222:30:062
305719940,4cyclictest0-21swapper/222:27:552
305719940,4cyclictest0-21swapper/222:10:132
305719940,4cyclictest0-21swapper/222:01:062
305719940,4cyclictest0-21swapper/221:55:392
305719940,4cyclictest0-21swapper/221:10:102
305719940,4cyclictest0-21swapper/220:50:242
305719940,4cyclictest0-21swapper/219:45:182
305719940,4cyclictest0-21swapper/219:20:132
305719940,4cyclictest0-21swapper/219:20:122
305719940,4cyclictest0-21swapper/200:35:122
305719940,4cyclictest0-21swapper/200:31:222
305719940,4cyclictest0-21swapper/200:26:012
305719940,4cyclictest0-21swapper/200:15:332
305719940,4cyclictest0-21swapper/200:01:462
305719940,3cyclictest4431-21ssh22:21:162
305719940,3cyclictest3573-21cut23:10:142
305719940,3cyclictest30259-21ssh23:23:282
305719940,3cyclictest12380-21apt-get00:20:012
305719940,3cyclictest0-21swapper/219:10:122
305639944,0cyclictest0-21swapper/123:42:201
305639944,0cyclictest0-21swapper/122:56:111
305639944,0cyclictest0-21swapper/119:15:021
305639942,1cyclictest0-21swapper/122:10:121
305639941,2cyclictest0-21swapper/123:50:121
305639941,2cyclictest0-21swapper/123:25:121
305639941,2cyclictest0-21swapper/123:10:151
305639941,2cyclictest0-21swapper/122:05:141
305639941,2cyclictest0-21swapper/121:35:141
305639941,2cyclictest0-21swapper/121:30:151
305639941,2cyclictest0-21swapper/121:20:121
305639941,2cyclictest0-21swapper/120:40:111
305639941,2cyclictest0-21swapper/120:35:121
305639941,2cyclictest0-21swapper/120:25:121
305639941,2cyclictest0-21swapper/120:15:121
305639941,2cyclictest0-21swapper/100:25:111
305639941,2cyclictest0-21swapper/100:15:141
305639941,1cyclictest0-21swapper/119:35:121
305639940,4cyclictest0-21swapper/122:50:141
305639940,4cyclictest0-21swapper/121:58:181
305599944,0cyclictest0-21swapper/022:02:320
305599944,0cyclictest0-21swapper/021:15:140
305599943,1cyclictest8326-21grep20:30:190
305599943,1cyclictest693-21kworker/0:022:58:450
305599943,1cyclictest693-21kworker/0:022:50:100
305599943,1cyclictest693-21kworker/0:022:24:540
305599943,1cyclictest693-21kworker/0:021:50:490
305599943,1cyclictest693-21kworker/0:021:06:410
305599943,1cyclictest693-21kworker/0:021:04:290
305599943,1cyclictest693-21kworker/0:020:56:240
305599943,1cyclictest693-21kworker/0:020:53:440
305599943,1cyclictest693-21kworker/0:020:11:210
305599943,1cyclictest693-21kworker/0:019:58:080
305599943,1cyclictest16773-21cpuspeed19:45:130
305599942,2cyclictest693-21kworker/0:023:50:210
305599942,2cyclictest693-21kworker/0:019:35:010
305599942,2cyclictest693-21kworker/0:019:25:000
305599942,1cyclictest714-21snmpd20:35:040
305599941,3cyclictest693-21kworker/0:023:10:170
305599941,2cyclictest0-21swapper/022:35:130
305599941,2cyclictest0-21swapper/022:35:130
305599940,4cyclictest19214-21apt-get19:50:140
305599940,4cyclictest0-21swapper/023:43:480
305599940,4cyclictest0-21swapper/023:15:120
305599940,4cyclictest0-21swapper/023:05:040
305599940,4cyclictest0-21swapper/023:00:130
305599940,4cyclictest0-21swapper/020:25:190
305599940,4cyclictest0-21swapper/019:30:120
305599940,4cyclictest0-21swapper/000:10:150
305599940,3cyclictest8912-21ssh23:45:160
305599940,3cyclictest5944-21ssh21:49:150
305599940,3cyclictest4960-21irqstats19:20:160
305599940,3cyclictest4960-21irqstats19:20:160
305599940,3cyclictest4763-21ntp_states22:05:200
305599940,3cyclictest4562-21sh21:31:510
305599940,3cyclictest29122-21/usr/sbin/munin23:55:180
305599940,3cyclictest23060-21dpkg00:25:110
305599940,3cyclictest19843-21memory22:45:160
305599940,3cyclictest16370-21grep20:50:000
305599940,3cyclictest15711-21users19:40:210
305599940,3cyclictest15219-21sh21:37:160
305599940,3cyclictest13700-21df00:20:150
305599940,3cyclictest12079-21df20:40:130
305599940,3cyclictest10561-21/usr/sbin/munin23:30:100
305599940,3cyclictest0-21swapper/020:20:110
305599940,1cyclictest20159-21/usr/sbin/munin21:40:170
305599940,1cyclictest15503-21/usr/sbin/munin00:05:170
305599940,0cyclictest0-21swapper/020:00:000
30024240,1sleep20-21swapper/221:45:102
28410240,3sleep00-21swapper/022:33:530
305769931,1cyclictest12558-21uname20:40:163
305769930,3cyclictest4019-21basename19:20:133
305769930,3cyclictest4019-21basename19:20:133
305769930,3cyclictest2737-21ssh00:15:033
305769930,3cyclictest0-21swapper/321:30:123
305769930,3cyclictest0-21swapper/321:00:113
305769930,3cyclictest0-21swapper/320:50:163
305769930,3cyclictest0-21swapper/320:45:123
305769930,3cyclictest0-21swapper/320:30:103
305769930,3cyclictest0-21swapper/320:25:113
305769930,3cyclictest0-21swapper/320:20:133
305769930,3cyclictest0-21swapper/320:15:063
305769930,3cyclictest0-21swapper/320:10:133
305769930,3cyclictest0-21swapper/320:05:083
305769930,3cyclictest0-21swapper/319:55:153
305769930,3cyclictest0-21swapper/319:50:023
305769930,3cyclictest0-21swapper/319:35:113
305769930,3cyclictest0-21swapper/319:30:093
305769930,3cyclictest0-21swapper/319:10:123
305769930,1cyclictest5213-21ssh00:00:013
305719931,1cyclictest0-21swapper/219:40:112
305719930,3cyclictest8911-21apt-get19:30:132
305719930,3cyclictest32578-21irqstats20:15:172
305719930,3cyclictest30551-21apt-config22:35:022
305719930,3cyclictest30551-21apt-config22:35:022
305719930,3cyclictest24532-21diskmemload23:55:012
305719930,3cyclictest22733-21ntp_states21:00:172
305719930,3cyclictest19340-21ls20:55:132
305719930,3cyclictest18390-21taskset22:45:012
305719930,3cyclictest1285-21diskstats19:15:012
305719930,3cyclictest10569-21apt-get23:30:102
305719930,3cyclictest0-21swapper/223:40:142
305719930,3cyclictest0-21swapper/222:05:052
305719930,3cyclictest0-21swapper/221:40:072
305719930,3cyclictest0-21swapper/221:30:122
305719930,3cyclictest0-21swapper/221:25:122
305719930,3cyclictest0-21swapper/221:15:142
305719930,3cyclictest0-21swapper/220:45:122
305719930,3cyclictest0-21swapper/220:40:102
305719930,3cyclictest0-21swapper/220:35:112
305719930,3cyclictest0-21swapper/220:30:112
305719930,3cyclictest0-21swapper/220:25:042
305719930,3cyclictest0-21swapper/220:20:032
305719930,3cyclictest0-21swapper/220:10:022
305719930,3cyclictest0-21swapper/220:00:092
305719930,3cyclictest0-21swapper/219:55:112
305719930,3cyclictest0-21swapper/219:50:022
305719930,3cyclictest0-21swapper/219:35:112
305719930,3cyclictest0-21swapper/219:25:092
305719930,2cyclictest24068-21dpkg21:05:012
305639931,1cyclictest0-21swapper/121:15:131
305639931,1cyclictest0-21swapper/121:10:011
305639931,1cyclictest0-21swapper/120:45:111
305639931,1cyclictest0-21swapper/120:05:131
305639931,1cyclictest0-21swapper/119:55:111
305639931,1cyclictest0-21swapper/119:50:131
305639931,1cyclictest0-21swapper/119:25:091
305639931,1cyclictest0-21swapper/119:10:131
305639931,1cyclictest0-21swapper/100:35:111
305639931,1cyclictest0-21swapper/100:00:101
305639930,2cyclictest0-21swapper/122:40:131
305639930,2cyclictest0-21swapper/122:30:121
305639930,2cyclictest0-21swapper/122:05:001
305639930,2cyclictest0-21swapper/121:10:091
305639930,2cyclictest0-21swapper/120:10:121
305599932,1cyclictest714-21snmpd20:05:040
305599932,1cyclictest693-21kworker/0:019:15:100
305599931,2cyclictest23093-21sh22:15:010
305599931,1cyclictest0-21swapper/022:25:120
305599930,3cyclictest618-21ssh23:25:070
305599930,3cyclictest10473-21sh00:35:090
305599930,3cyclictest0-21swapper/023:35:130
305599930,3cyclictest0-21swapper/000:00:080
23272230,0sleep10-21swapper/123:20:131
305639921,1cyclictest0-21swapper/123:35:141
305639921,1cyclictest0-21swapper/120:30:111
305639920,2cyclictest0-21swapper/120:00:101
305639920,1cyclictest20994-21chrt20:57:281
305639920,1cyclictest0-21swapper/123:15:121
305639920,1cyclictest0-21swapper/123:05:141
305639920,1cyclictest0-21swapper/123:00:141
305639920,1cyclictest0-21swapper/122:35:021
305639920,1cyclictest0-21swapper/122:35:011
305639920,1cyclictest0-21swapper/121:50:011
305639920,1cyclictest0-21swapper/121:00:121
305639920,1cyclictest0-21swapper/120:50:151
305639920,1cyclictest0-21swapper/119:45:011
305639920,1cyclictest0-21swapper/119:30:131
305639920,1cyclictest0-21swapper/100:05:151
305639910,1cyclictest0-21swapper/120:19:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional