You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 09:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Thu Nov 27, 2025 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
89862205189,11sleep10-21swapper/119:08:431
88142197180,12sleep30-21swapper/319:06:303
87642187169,13sleep00-21swapper/019:05:500
8962210486,13sleep20-21swapper/219:08:242
65862440,0sleep20-21swapper/223:55:092
114072410,0sleep10-21swapper/121:15:071
65622400,0sleep10-21swapper/100:27:031
65622400,0sleep10-21swapper/100:27:031
196802390,0sleep20-21swapper/223:12:172
294412120,0sleep10-21swapper/122:11:591
924699113,4cyclictest24255-21awk22:10:000
2655290,0sleep10-21swapper/123:20:121
2058290,0sleep10-21swapper/123:52:231
92559970,6cyclictest0-21swapper/220:20:102
92559960,1cyclictest22553-21run-parts20:45:182
92469960,3cyclictest19195-21perf22:40:000
92599955,0cyclictest11827-21kworker/3:200:34:353
92599950,5cyclictest0-21swapper/322:28:093
92599950,5cyclictest0-21swapper/320:55:013
92599950,5cyclictest0-21swapper/300:07:143
92599950,4cyclictest21574-21sh23:13:293
92599950,4cyclictest1828-21/usr/sbin/munin00:25:183
92599950,4cyclictest1828-21/usr/sbin/munin00:25:173
92599950,4cyclictest16935-21apt-get21:50:123
92599950,2cyclictest0-21swapper/300:15:153
92559950,5cyclictest29387-21rm23:00:302
92559950,5cyclictest0-21swapper/223:41:222
92559950,5cyclictest0-21swapper/223:17:302
92559950,5cyclictest0-21swapper/222:56:122
92559950,5cyclictest0-21swapper/222:30:152
92559950,0cyclictest3831-21ssh23:36:522
92509955,0cyclictest0-21swapper/123:31:351
92509950,5cyclictest0-21swapper/123:48:101
92509950,5cyclictest0-21swapper/120:25:021
92509950,5cyclictest0-21swapper/100:15:211
92509950,5cyclictest0-21swapper/100:03:181
92469955,0cyclictest0-21swapper/023:59:060
92469954,1cyclictest21477-21rm23:13:250
92469951,4cyclictest32193-21basename19:55:010
92469951,2cyclictest0-21swapper/021:45:120
92469950,5cyclictest0-21swapper/023:04:320
92469950,5cyclictest0-21swapper/021:30:330
92469950,5cyclictest0-21swapper/021:26:550
92469950,5cyclictest0-21swapper/019:30:110
92469950,5cyclictest0-21swapper/000:07:240
92469950,4cyclictest30206-21hddtemp_smartct21:40:130
92599944,0cyclictest11827-21kworker/3:223:16:123
92599944,0cyclictest0-21swapper/323:47:363
92599944,0cyclictest0-21swapper/300:35:193
92599943,1cyclictest24770-21/usr/sbin/munin22:10:193
92599942,1cyclictest21056-21/usr/sbin/munin20:45:113
92599942,1cyclictest15105-21/usr/sbin/munin19:20:163
92599941,2cyclictest0-21swapper/322:30:153
92599941,1cyclictest0-21swapper/320:50:103
92599941,1cyclictest0-21swapper/320:00:133
92599940,4cyclictest21604-21apt-get21:20:103
92599940,4cyclictest18098-21df_inode19:25:133
92599940,4cyclictest15559-21perl22:05:153
92599940,4cyclictest12132-21ssh23:40:503
92599940,4cyclictest0-21swapper/323:30:143
92599940,4cyclictest0-21swapper/323:26:463
92599940,4cyclictest0-21swapper/323:20:093
92599940,4cyclictest0-21swapper/323:05:103
92599940,4cyclictest0-21swapper/322:51:123
92599940,4cyclictest0-21swapper/321:33:533
92599940,4cyclictest0-21swapper/320:34:583
92599940,4cyclictest0-21swapper/300:20:023
92599940,4cyclictest0-21swapper/300:10:113
92599940,3cyclictest9273-21sh22:01:533
92599940,3cyclictest8367-21ntp_states21:45:183
92599940,3cyclictest8283-21sh22:17:473
92599940,3cyclictest7421-21/usr/sbin/munin20:10:223
92599940,3cyclictest5095-21ssh23:37:403
92599940,3cyclictest32226-21ntp_states21:05:183
92599940,3cyclictest30165-21hddtemp_smartct23:50:173
92599940,3cyclictest29557-21tr21:40:113
92599940,3cyclictest28338-21/usr/sbin/munin19:45:143
92599940,3cyclictest26820-21apt-config21:55:013
92599940,3cyclictest26680-21sh22:43:273
92599940,3cyclictest2640-21ssh21:10:173
92599940,3cyclictest22810-21ssh21:36:163
92599940,3cyclictest20251-21ssh00:01:133
92599940,3cyclictest1832-21sh23:03:353
92599940,3cyclictest18111-21sh21:18:083
92599940,3cyclictest17939-21ssh22:39:153
92599940,3cyclictest10741-21hwlatdetect20:15:163
92599940,3cyclictest10399-21sh23:56:243
92599940,3cyclictest0-21swapper/322:20:123
92599940,1cyclictest29764-21ntp_states21:00:183
92599940,1cyclictest12388-21/usr/sbin/munin20:20:103
92559944,0cyclictest0-21swapper/223:45:522
92559944,0cyclictest0-21swapper/200:15:172
92559943,1cyclictest0-21swapper/223:09:222
92559942,2cyclictest714-21snmpd19:24:312
92559942,1cyclictest15664-21ssh22:21:082
92559941,3cyclictest714-21snmpd22:44:442
92559941,2cyclictest13507-21cat19:15:152
92559940,4cyclictest26093-21memory19:40:152
92559940,4cyclictest1835-21apt-get00:25:132
92559940,4cyclictest1835-21apt-get00:25:122
92559940,4cyclictest0-21swapper/223:25:142
92559940,4cyclictest0-21swapper/223:22:072
92559940,4cyclictest0-21swapper/222:45:122
92559940,4cyclictest0-21swapper/222:36:312
92559940,4cyclictest0-21swapper/222:15:112
92559940,4cyclictest0-21swapper/222:14:142
92559940,4cyclictest0-21swapper/222:02:122
92559940,4cyclictest0-21swapper/221:45:122
92559940,4cyclictest0-21swapper/221:40:112
92559940,4cyclictest0-21swapper/221:35:112
92559940,4cyclictest0-21swapper/221:29:072
92559940,4cyclictest0-21swapper/221:15:122
92559940,4cyclictest0-21swapper/221:10:122
92559940,4cyclictest0-21swapper/220:30:062
92559940,4cyclictest0-21swapper/220:05:112
92559940,4cyclictest0-21swapper/219:35:122
92559940,4cyclictest0-21swapper/200:05:002
92559940,4cyclictest0-21swapper/200:02:432
92559940,3cyclictest9873-21sed20:15:012
92559940,3cyclictest8993-21ssh21:30:022
92559940,3cyclictest32378-21sed19:55:012
92559940,3cyclictest31019-21ssh21:56:282
92559940,3cyclictest30737-21/usr/sbin/munin21:05:012
92559940,3cyclictest25706-21ssh00:36:332
92559940,3cyclictest24742-21rm22:25:352
92559940,3cyclictest19981-21cat19:30:012
92559940,3cyclictest1017-21ssh23:51:452
92559940,3cyclictest0-21swapper/223:30:012
92559940,3cyclictest0-21swapper/222:50:122
92509944,0cyclictest0-21swapper/123:55:441
92509944,0cyclictest0-21swapper/123:02:371
92509944,0cyclictest0-21swapper/121:31:241
92509944,0cyclictest0-21swapper/100:34:261
92509942,1cyclictest0-21swapper/100:10:111
92509941,2cyclictest0-21swapper/123:35:111
92509941,2cyclictest0-21swapper/123:25:131
92509941,1cyclictest0-21swapper/122:45:131
92509941,1cyclictest0-21swapper/122:40:101
92509941,1cyclictest0-21swapper/122:05:121
92509941,1cyclictest0-21swapper/121:25:111
92509940,4cyclictest0-21swapper/122:15:201
92509940,0cyclictest0-21swapper/122:27:501
92509940,0cyclictest0-21swapper/121:57:441
92509940,0cyclictest0-21swapper/100:39:481
92469944,0cyclictest0-21swapper/023:30:260
92469943,1cyclictest11266-21munin-run00:30:010
92469943,1cyclictest11266-21munin-run00:30:000
92469942,1cyclictest30737-21munin-run21:05:020
92469942,1cyclictest1816-21ssh23:20:040
92469941,2cyclictest0-21swapper/020:40:080
92469941,1cyclictest0-21swapper/020:50:110
92469940,4cyclictest767-21diskmemload00:15:120
92469940,4cyclictest4953-21apt-get20:05:120
92469940,4cyclictest1631-21chrt22:46:490
92469940,4cyclictest0-21swapper/023:50:140
92469940,4cyclictest0-21swapper/023:25:140
92469940,4cyclictest0-21swapper/022:50:130
92469940,4cyclictest0-21swapper/022:33:490
92469940,4cyclictest0-21swapper/022:25:120
92469940,4cyclictest0-21swapper/022:00:140
92469940,4cyclictest0-21swapper/021:50:120
92469940,4cyclictest0-21swapper/021:37:270
92469940,4cyclictest0-21swapper/021:05:130
92469940,4cyclictest0-21swapper/019:45:120
92469940,4cyclictest0-21swapper/000:35:130
92469940,4cyclictest0-21swapper/000:30:160
92469940,4cyclictest0-21swapper/000:20:110
92469940,4cyclictest0-21swapper/000:10:110
92469940,3cyclictest3475-21irqstats20:00:180
92469940,3cyclictest31495-21chrt23:35:010
92469940,3cyclictest30951-21sh23:18:080
92469940,3cyclictest26448-21apt-get21:55:010
92469940,3cyclictest22050-21irqstats20:45:140
92469940,3cyclictest21740-21cpu21:20:120
92469940,3cyclictest12451-21memory19:15:010
92469940,3cyclictest11799-21cron22:20:010
92469940,3cyclictest0-21swapper/022:55:120
22397240,0sleep022426-21ssh22:40:450
92599931,2cyclictest55950irq/130-eth022:55:103
92599931,1cyclictest0-21swapper/319:30:113
92599930,3cyclictest5482-21diskstats20:05:143
92599930,3cyclictest23499-21irqstats19:35:173
92599930,3cyclictest17970-21chrt20:35:283
92599930,3cyclictest12385-21grep19:15:013
92599930,3cyclictest0-21swapper/322:45:133
92599930,3cyclictest0-21swapper/321:25:043
92599930,3cyclictest0-21swapper/320:40:113
92599930,3cyclictest0-21swapper/320:25:043
92599930,3cyclictest0-21swapper/319:55:133
92599930,3cyclictest0-21swapper/319:50:123
92599930,3cyclictest0-21swapper/319:40:113
92599930,3cyclictest0-21swapper/319:10:123
92559932,1cyclictest714-21snmpd00:20:082
92559932,1cyclictest4314-21ssh00:10:072
92559932,1cyclictest11714-21ssh00:30:092
92559931,1cyclictest0-21swapper/219:10:112
92559930,3cyclictest26266-21expr20:55:102
92559930,3cyclictest16854-21ssh21:50:082
92559930,3cyclictest0-21swapper/222:05:062
92559930,3cyclictest0-21swapper/220:50:112
92559930,3cyclictest0-21swapper/220:40:082
92559930,3cyclictest0-21swapper/220:35:372
92559930,3cyclictest0-21swapper/220:25:042
92559930,3cyclictest0-21swapper/220:15:132
92559930,3cyclictest0-21swapper/219:55:062
92559930,3cyclictest0-21swapper/219:45:062
92559930,3cyclictest0-21swapper/219:30:112
92559930,2cyclictest30955-21/usr/sbin/munin21:05:142
92559930,2cyclictest2483-21/usr/sbin/munin20:00:132
92559930,2cyclictest21457-21ssh21:20:052
92509933,0cyclictest0-21swapper/123:10:031
92509933,0cyclictest0-21swapper/122:55:551
92509933,0cyclictest0-21swapper/121:09:241
92509933,0cyclictest0-21swapper/120:48:451
92509933,0cyclictest0-21swapper/119:42:591
92509931,1cyclictest0-21swapper/123:40:101
92509931,1cyclictest0-21swapper/123:15:111
92509931,1cyclictest0-21swapper/122:50:131
92509931,1cyclictest0-21swapper/122:35:131
92509931,1cyclictest0-21swapper/122:20:121
92509931,1cyclictest0-21swapper/122:00:131
92509931,1cyclictest0-21swapper/121:50:121
92509931,1cyclictest0-21swapper/121:20:111
92509931,1cyclictest0-21swapper/121:10:111
92509931,1cyclictest0-21swapper/120:40:071
92509931,1cyclictest0-21swapper/120:30:071
92509931,1cyclictest0-21swapper/120:05:121
92509931,1cyclictest0-21swapper/119:25:111
92509931,1cyclictest0-21swapper/119:20:131
92509931,1cyclictest0-21swapper/119:10:121
92509931,1cyclictest0-21swapper/100:20:121
92509931,1cyclictest0-21swapper/100:05:111
92509930,3cyclictest0-21swapper/119:57:141
92509930,3cyclictest0-21swapper/119:52:431
92509930,2cyclictest0-21swapper/123:05:111
92509930,2cyclictest0-21swapper/121:40:111
92509930,0cyclictest0-21swapper/120:57:371
92509930,0cyclictest0-21swapper/119:33:581
92469931,1cyclictest0-21swapper/020:20:090
92469930,3cyclictest9550-21cat19:10:130
92469930,3cyclictest9465-21grep23:40:110
92469930,3cyclictest767-21diskmemload00:00:000
92469930,3cyclictest19262-21apt-get23:45:110
92469930,3cyclictest17573-21apt-get19:25:100
92469930,3cyclictest12788-21expr22:20:130
92469930,3cyclictest0-21swapper/023:05:040
92469930,3cyclictest0-21swapper/022:10:120
92469930,3cyclictest0-21swapper/021:15:100
92469930,3cyclictest0-21swapper/021:10:020
92469930,3cyclictest0-21swapper/020:55:100
92469930,3cyclictest0-21swapper/020:38:110
92469930,3cyclictest0-21swapper/020:30:070
92469930,3cyclictest0-21swapper/020:25:060
92469930,3cyclictest0-21swapper/020:15:130
92469930,3cyclictest0-21swapper/020:10:130
92469930,3cyclictest0-21swapper/019:55:130
92469930,3cyclictest0-21swapper/019:40:110
92469930,3cyclictest0-21swapper/019:35:130
92469930,3cyclictest0-21swapper/019:20:140
92509921,1cyclictest0-21swapper/122:30:141
92509921,1cyclictest0-21swapper/120:10:121
92509921,1cyclictest0-21swapper/119:45:101
92509921,0cyclictest0-21swapper/120:15:121
92509920,2cyclictest17993-21chrt20:35:451
92509920,2cyclictest0-21swapper/121:45:121
92509920,2cyclictest0-21swapper/121:35:101
92509920,1cyclictest14500-21chrt19:15:431
92509920,1cyclictest0-21swapper/121:00:111
92509920,1cyclictest0-21swapper/120:50:111
92509920,1cyclictest0-21swapper/120:20:101
92509920,1cyclictest0-21swapper/120:00:011
92509920,1cyclictest0-21swapper/119:35:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional