You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 09:30
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8.osadl.org (updated Fri Nov 07, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11406352580,1sleep00-21swapper/023:00:160
818590991916,1cyclictest0-21swapper/323:05:003
818590991815,1cyclictest0-21swapper/300:30:223
818590991714,1cyclictest0-21swapper/322:55:013
818590991714,1cyclictest0-21swapper/321:50:233
818590991714,1cyclictest0-21swapper/300:20:213
818590991613,1cyclictest0-21swapper/323:35:173
818590991613,1cyclictest0-21swapper/321:40:143
818590991613,1cyclictest0-21swapper/319:30:283
10184552161,4sleep01018514-21ntp_states21:40:210
81859099152,9cyclictest12237922sleep323:52:133
818590991512,1cyclictest0-21swapper/321:30:163
818590991512,1cyclictest0-21swapper/321:20:183
818590991512,1cyclictest0-21swapper/319:15:013
818590991512,1cyclictest0-21swapper/300:35:163
818590991412,1cyclictest0-21swapper/323:55:143
818590991411,1cyclictest0-21swapper/323:45:193
818590991411,1cyclictest0-21swapper/323:20:183
818590991411,1cyclictest0-21swapper/323:15:183
818590991411,1cyclictest0-21swapper/323:10:183
818590991411,1cyclictest0-21swapper/323:05:173
818590991411,1cyclictest0-21swapper/321:55:233
818590991411,1cyclictest0-21swapper/321:45:213
818590991411,1cyclictest0-21swapper/320:50:163
818590991411,1cyclictest0-21swapper/320:21:583
818590991411,1cyclictest0-21swapper/320:15:223
818590991411,1cyclictest0-21swapper/320:10:023
818590991411,1cyclictest0-21swapper/319:10:003
818590991411,1cyclictest0-21swapper/300:08:233
81859099140,12cyclictest0-21swapper/322:04:563
174698140,3rtkit-daemon1123397-21ssh22:50:081
174698140,1rtkit-daemon0-21swapper/123:03:561
81859099137,3cyclictest887540-21turbostat20:05:003
81859099131,9cyclictest1012456-21taskset21:38:313
81859099131,8cyclictest971517-21taskset21:10:223
81859099131,8cyclictest971517-21taskset21:10:213
818590991311,1cyclictest0-21swapper/320:59:483
818590991310,1cyclictest0-21swapper/323:30:203
818590991310,1cyclictest0-21swapper/322:34:593
818590991310,1cyclictest0-21swapper/322:25:163
818590991310,1cyclictest0-21swapper/322:10:173
818590991310,1cyclictest0-21swapper/321:10:003
818590991310,1cyclictest0-21swapper/320:35:153
818590991310,1cyclictest0-21swapper/320:30:213
818590991310,1cyclictest0-21swapper/319:40:013
818590991310,1cyclictest0-21swapper/300:27:203
818590991310,1cyclictest0-21swapper/300:15:003
12635562130,1sleep30-21swapper/300:17:383
12021992131,2sleep00-21swapper/023:40:160
11315842131,4sleep01131686-21tail22:55:150
81859099129,1cyclictest0-21swapper/323:40:193
81859099129,1cyclictest0-21swapper/322:55:163
81859099129,1cyclictest0-21swapper/322:43:333
81859099129,1cyclictest0-21swapper/322:08:543
81859099129,1cyclictest0-21swapper/321:30:003
81859099121,8cyclictest869109-21taskset19:48:153
81859099121,8cyclictest859047-21taskset19:40:173
81859099121,8cyclictest838487-21taskset19:20:263
81859099121,8cyclictest1240784-21taskset00:04:233
81859099121,8cyclictest1121127-21taskset22:45:293
81859099121,8cyclictest1105561-21taskset22:35:433
818590991210,1cyclictest0-21swapper/322:15:183
818590991210,1cyclictest0-21swapper/320:49:593
818590991210,1cyclictest0-21swapper/320:45:013
81859099118,1cyclictest0-21swapper/322:20:173
81859099118,1cyclictest0-21swapper/319:30:003
81859099118,1cyclictest0-21swapper/319:15:153
81859099111,8cyclictest917793-21taskset20:28:073
81859099111,7cyclictest954682-21taskset21:00:173
81859099110,8cyclictest881232-21taskset19:56:413
81859099110,8cyclictest1183866-21taskset23:25:273
174698110,7rtkit-daemon1745-21rtkit-daemon19:56:430
174698110,2rtkit-daemon275-21plymouthd19:44:130
10685512111,6sleep01068584-21apt-get22:15:130
9792302100,8sleep0979259-21/usr/sbin/munin21:15:240
9723362101,7sleep1972342-21ssh21:11:261
9723362101,7sleep1972342-21ssh21:11:251
9412852100,4sleep0941402-21timerandwakeup20:45:240
8935232102,5sleep0893587-21unin-run20:10:000
8199342100,1sleep0820986-21grep19:10:170
81859099107,1cyclictest0-21swapper/321:15:163
81859099101,7cyclictest899443-21taskset20:10:343
81859099101,6cyclictest873533-21taskset19:50:193
174698100,5rtkit-daemon972554-21ssh21:11:520
174698100,5rtkit-daemon972554-21ssh21:11:510
174698100,2rtkit-daemon275-21plymouthd00:23:101
12471072100,7sleep11247115-21ssh00:05:471
997775290,6sleep1997814-21grep21:30:131
997737290,1sleep00-21swapper/021:30:120
17469890,5rtkit-daemon894229-21df_inode20:10:140
17469890,5rtkit-daemon824900-21munin-node23:20:151
17469890,5rtkit-daemon771-21snmpd23:12:281
17469890,5rtkit-daemon1798-21sendmail-mta19:38:100
17469890,5rtkit-daemon1199663-21ssh23:35:451
17469890,4rtkit-daemon982375-21perl21:20:151
17469890,4rtkit-daemon1170178-21awk23:20:001
17469890,3rtkit-daemon275-21plymouthd21:40:141
17469890,1rtkit-daemon0-21swapper/123:58:401
17469890,1rtkit-daemon0-21swapper/123:51:561
17469890,1rtkit-daemon0-21swapper/100:26:281
17469890,1rtkit-daemon0-21swapper/020:55:400
17469890,1rtkit-daemon0-21swapper/020:00:290
17469890,1rtkit-daemon0-21swapper/019:30:010
1249591290,6sleep21249719-21cut00:10:162
1089164291,1sleep20-21swapper/222:25:212
1082251291,1sleep00-21swapper/022:21:100
1037803290,6sleep01038802-21grep21:55:170
925481280,5sleep2926884-21cat20:35:192
881771281,3sleep2881804-21apt-get20:00:132
826954281,3sleep00-21swapper/019:15:110
17469880,5rtkit-daemon275-21plymouthd19:25:230
17469880,4rtkit-daemon966060-21diskmemload22:11:131
17469880,4rtkit-daemon966060-21diskmemload00:17:281
17469880,4rtkit-daemon824900-21munin-node23:31:051
17469880,4rtkit-daemon824900-21munin-node21:51:101
17469880,4rtkit-daemon824900-21munin-node21:26:291
17469880,4rtkit-daemon824900-21munin-node00:32:111
17469880,4rtkit-daemon824900-21munin-node00:03:481
17469880,4rtkit-daemon774-21ntpd21:57:441
17469880,4rtkit-daemon551-21gmain22:17:001
17469880,4rtkit-daemon499-21avahi-daemon00:10:031
17469880,4rtkit-daemon1798-21sendmail-mta19:09:450
17469880,4rtkit-daemon1-21systemd20:40:450
17469880,4rtkit-daemon1215970-21sh23:47:121
17469880,4rtkit-daemon1050399-21ssh22:00:321
17469880,4rtkit-daemon0-21swapper/020:28:490
17469880,3rtkit-daemon508-21rsyslogd19:20:060
17469880,3rtkit-daemon1-21systemd23:40:181
17469880,3rtkit-daemon1-21systemd20:31:260
17469880,3rtkit-daemon1-21systemd19:52:070
17469880,3rtkit-daemon1137577-21ssh22:57:081
17469880,1rtkit-daemon0-21swapper/122:43:051
17469880,1rtkit-daemon0-21swapper/021:06:260
17469880,1rtkit-daemon0-21swapper/021:00:280
17469880,1rtkit-daemon0-21swapper/020:35:080
17469880,1rtkit-daemon0-21swapper/020:20:460
1295883280,5sleep01295955-21unin-run00:39:590
1200236280,5sleep01200244-21ssh23:37:080
1162638281,3sleep21162673-21apt-get23:15:132
1138700281,5sleep21138706-21ssh22:59:462
1109654281,4sleep01111616-21grep22:40:180
1098860280,2sleep01098868-21ssh22:33:200
982098270,4sleep0982259-21diskstats21:20:140
965971270,1sleep10-21swapper/121:09:011
950261270,4sleep20-21swapper/220:55:192
930598270,5sleep1931479-21cut20:40:201
912150271,3sleep2912183-21apt-get20:25:132
899786271,3sleep2899808-21munin-run20:14:592
887517270,4sleep1887579-21sed20:05:001
877061270,5sleep1878184-21kthreadcore19:55:201
857448271,2sleep1857481-21apt-get19:40:121
845136270,3sleep2845169-21apt-get19:30:152
827143271,3sleep2827266-21hddtemp_smartct19:15:122
824691270,4sleep2824884-21latency_hist19:15:012
17469870,3rtkit-daemon824900-21munin-node22:36:431
17469870,3rtkit-daemon824900-21munin-node00:35:161
17469870,3rtkit-daemon771-21snmpd21:36:281
17469870,3rtkit-daemon3646-21packagekitd22:06:181
17469870,3rtkit-daemon275-21plymouthd22:46:221
17469870,3rtkit-daemon1-21systemd23:05:581
17469870,3rtkit-daemon1-21systemd22:26:541
17469870,3rtkit-daemon1-21systemd21:45:061
17469870,3rtkit-daemon1-21systemd20:50:140
17469870,3rtkit-daemon1-21systemd20:15:500
17469870,3rtkit-daemon1-21systemd19:45:430
17469870,3rtkit-daemon1076391-21munin-node22:20:111
17469870,2rtkit-daemon275-21plymouthd23:25:091
1255214270,1sleep00-21swapper/000:11:190
1247848270,1sleep00-21swapper/000:07:250
1229798271,4sleep01230400-21memory23:55:200
1193984271,3sleep21194017-21apt-get23:35:142
1168536270,1sleep00-21swapper/023:16:200
1152887270,1sleep00-21swapper/023:06:260
1122327270,5sleep01122329-21ssh22:48:140
1092388270,4sleep11093260-21sed22:30:171
1070917271,3sleep21072931-21grep22:15:192
1066548273,2sleep00-21swapper/022:11:390
1050082270,1sleep00-21swapper/022:00:250
1044363270,1sleep20-21swapper/221:59:442
1034962270,1sleep00-21swapper/021:51:110
1019072270,1sleep20-21swapper/221:40:322
1011445270,4sleep01011453-21ssh21:36:090
994835260,3sleep0994915-21ntp_states21:25:220
974136260,1sleep20-21swapper/221:15:062
947900260,1sleep20-21swapper/220:53:512
944253260,4sleep1945086-21kthreadcore20:50:221
936598260,4sleep1937409-21irqrtprio20:45:161
936271261,3sleep2936380-21tr20:45:132
918202261,2sleep2918235-21apt-get20:30:152
916842261,3sleep1917003-21grep20:25:201
911577260,1sleep10-21swapper/120:21:001
865662260,3sleep2867735-21cut19:45:192
864531260,4sleep1865660-21grep19:45:181
833921260,4sleep2834856-21cut19:20:182
833377260,3sleep1833914-21ps19:20:161
832492260,1sleep10-21swapper/119:16:281
1287035260,1sleep00-21swapper/000:32:430
1279570260,3sleep21279578-21ssh00:28:312
1274102260,4sleep01274142-21ssh00:25:160
1272005261,3sleep21272011-21ssh00:23:512
1264983260,3sleep01265138-21df_inode00:20:170
1262659260,1sleep00-21swapper/000:15:310
1241641260,4sleep21241753-21if_err_eno200:05:162
1240377260,1sleep00-21swapper/000:03:330
1235418260,3sleep2421rcuc/200:00:222
1223646260,1sleep00-21swapper/023:51:500
1208070260,1sleep20-21swapper/223:41:582
1186075260,1sleep00-21swapper/023:30:060
1170410260,1sleep20-21swapper/223:20:032
1170141260,1sleep00-21swapper/023:20:000
1160473260,4sleep00-21swapper/023:10:470
1152167260,4sleep21146940-21/usr/sbin/munin23:05:232
1129017260,1sleep00-21swapper/022:50:320
1124467260,3sleep21125614-21kthreadcore22:50:172
1107750260,2sleep21107789-21apt-get22:40:122
1089324260,1sleep00-21swapper/022:25:220
1060504260,3sleep21060649-21df22:10:132
1059935260,1sleep20-21swapper/222:09:342
1058381260,1sleep0275-21plymouthd22:05:590
1051814260,1sleep20-21swapper/222:03:482
972310250,3sleep2972318-21ssh21:11:232
972310250,3sleep2972318-21ssh21:11:222
959694250,3sleep1275-21plymouthd21:00:311
923916251,3sleep1331ktimers/120:34:551
900356250,3sleep1900469-21irqcore20:15:181
895859251,2sleep10-21swapper/120:10:181
893288251,3sleep2431ktimers/220:07:032
871509251,2sleep20-21swapper/219:50:192
869801251,2sleep10-21swapper/119:50:141
856309250,3sleep2856537-21users19:35:252
1286801250,3sleep20-21swapper/200:32:052
1264036250,3sleep21264038-21ssh00:18:462
1231721250,4sleep20-21swapper/223:56:502
1223567250,3sleep21223575-21ssh23:51:402
1215355251,2sleep2275-21plymouthd23:45:482
1186135250,2sleep20-21swapper/223:30:132
1179666250,2sleep00-21swapper/023:25:180
1178660251,2sleep20-21swapper/223:25:152
1162265251,2sleep20-21swapper/223:14:582
1139504250,3sleep21139725-21sed23:00:152
1115829250,3sleep21115951-21perl22:45:162
1097974250,3sleep21097982-21ssh22:31:132
1081523250,2sleep21081638-21find22:20:232
1035797250,3sleep21035799-21ssh21:53:112
1027708250,3sleep00-21swapper/021:47:320
1005602250,2sleep21005634-21apt-get21:35:102
1004060250,3sleep21004068-21ssh21:32:072
982382240,2sleep20-21swapper/221:20:162
974189240,1sleep10-21swapper/121:15:131
954601240,2sleep20-21swapper/221:00:172
906442240,2sleep20-21swapper/220:20:172
888649240,2sleep10-21swapper/120:05:211
858309240,2sleep20-21swapper/219:40:172
856182240,2sleep10-21swapper/119:35:241
844557240,1sleep10-21swapper/119:25:581
824530240,2sleep10-21swapper/119:12:561
818081240,2sleep2275-21plymouthd19:06:392
814352240,2sleep10-21swapper/119:05:171
1288409240,2sleep20-21swapper/200:35:122
1106946240,2sleep20-21swapper/222:38:552
1101299240,2sleep00-21swapper/022:35:200
1021270240,2sleep20-21swapper/221:45:122
989889230,1sleep20-21swapper/221:25:042
960292230,1sleep20-21swapper/221:05:052
949156230,1sleep10-21swapper/120:55:181
931480230,1sleep20-21swapper/220:40:212
924176230,1sleep10-21swapper/120:35:061
900035230,1sleep20-21swapper/220:15:042
875767230,1sleep20-21swapper/219:55:132
845415230,1sleep10-21swapper/119:30:181
839150230,1sleep20-21swapper/219:25:112
1209449230,1sleep00-21swapper/023:45:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional