You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-15 - 02:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8.osadl.org (updated Sun Apr 14, 2024 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
95078399759746,9cyclictest1203374-21kworker/0:1+events11:23:570
95078399756747,8cyclictest1221345-21kworker/0:1+events12:13:150
95078399755747,7cyclictest1127905-21kworker/0:2+events09:47:300
95078399749741,7cyclictest1226464-21kworker/0:0+events12:38:170
95078399748736,11cyclictest1198246-21kworker/0:0+events11:11:370
95078399747739,7cyclictest1198246-21kworker/0:0+events10:52:400
95078399747737,7cyclictest1198246-21kworker/0:0+events10:42:580
95078399746739,6cyclictest1221345-21kworker/0:1+events12:15:230
95078399745731,12cyclictest1068581-21kworker/0:1+events09:10:440
95078399743737,5cyclictest1068581-21kworker/0:1+events09:39:370
95078899742736,5cyclictest1081255-21kworker/1:1+events11:43:251
95078399742735,6cyclictest1191520-21kworker/0:1+events10:30:440
95078399741733,7cyclictest1127905-21kworker/0:2+events09:58:510
95078399741732,8cyclictest1203374-21kworker/0:1+events11:18:270
95078399740733,6cyclictest1221345-21kworker/0:1+events12:32:330
95079399739736,2cyclictest1184005-21turbostat10:14:553
95078399739733,5cyclictest1068581-21kworker/0:1+events09:30:230
95078399739732,6cyclictest1210942-21kworker/0:2+events11:31:060
95078399739732,6cyclictest1068581-21kworker/0:1+events09:15:230
95078399739731,7cyclictest925938-21kworker/0:2+events09:08:470
95078399739731,7cyclictest1221345-21kworker/0:1+events12:25:230
95078899738733,4cyclictest1081255-21kworker/1:1+events11:49:331
95078399738731,6cyclictest1221345-21kworker/0:1+events12:20:460
95078399738730,7cyclictest925938-21kworker/0:2+events08:50:220
95078399738730,7cyclictest1198246-21kworker/0:0+events11:02:560
95078399736729,6cyclictest925938-21kworker/0:2+events08:18:180
95078399734726,7cyclictest1127905-21kworker/0:2+events10:02:110
95078399733725,7cyclictest1198246-21kworker/0:0+events10:49:340
95078399732725,6cyclictest1068581-21kworker/0:1+events09:43:560
95079399731727,3cyclictest1050972-21turbostat08:34:553
95078399731724,6cyclictest925938-21kworker/0:2+events08:30:230
95078399731723,7cyclictest1068581-21kworker/0:1+events09:23:560
95079399727723,3cyclictest1191000-21turbostat10:19:553
95079399724720,3cyclictest1177004-21turbostat10:09:543
950793997240,722cyclictest1155971-21turbostat09:54:553
95078399723715,7cyclictest1191520-21kworker/0:1+events10:28:360
950793997220,721cyclictest995708-21turbostat07:49:563
950793997220,720cyclictest1226063-21turbostat12:09:553
95079099722718,3cyclictest1075092-21kworker/2:0+events09:53:152
95078399722714,7cyclictest925938-21kworker/0:2+events07:11:400
95078399720711,7cyclictest1191520-21kworker/0:1+events10:22:100
95078399719711,7cyclictest925938-21kworker/0:2+events08:28:490
95078399719709,9cyclictest1221345-21kworker/0:1+events12:05:430
95078399717710,6cyclictest1210942-21kworker/0:2+events11:39:250
95078399716709,6cyclictest1127905-21kworker/0:2+events10:05:210
95078399715708,6cyclictest1210942-21kworker/0:2+events11:52:120
95078399715707,7cyclictest1191520-21kworker/0:1+events10:36:060
950793997140,712cyclictest1075297-21turbostat08:54:553
95078399714707,6cyclictest1198246-21kworker/0:0+events10:55:210
95078399713706,6cyclictest925938-21kworker/0:2+events08:06:050
95078399713705,7cyclictest1162593-21kworker/0:0+events10:15:210
95078399712704,6cyclictest1210942-21kworker/0:2+events11:57:010
95078399711700,6cyclictest191rcu_preempt07:40:210
95078399708701,6cyclictest1162593-21kworker/0:0+events10:10:430
95078399707700,6cyclictest925938-21kworker/0:2+events07:18:150
95078399707698,8cyclictest1221345-21kworker/0:1+events12:02:540
95079399706703,1cyclictest0-21swapper/308:25:203
950793997050,703cyclictest1113944-21turbostat09:24:563
95078399705698,6cyclictest925938-21kworker/0:2+events07:58:160
95079399703700,1cyclictest0-21swapper/309:10:233
95078899703698,4cyclictest1216802-21turbostat11:39:551
95078399702692,9cyclictest925938-21kworker/0:2+events07:52:350
950793997010,699cyclictest1026108-21turbostat08:14:553
95078399700691,8cyclictest925938-21kworker/0:2+events08:35:220
95078399699692,6cyclictest925938-21kworker/0:2+events07:35:420
950793996980,696cyclictest1232166-21turbostat12:29:553
950793996970,695cyclictest1200575-21turbostat10:49:553
95079099697693,3cyclictest1148965-21turbostat09:49:552
95078399697690,6cyclictest925938-21kworker/0:2+events08:44:300
95078399697689,7cyclictest1198246-21kworker/0:0+events11:09:200
95078399697689,7cyclictest1068581-21kworker/0:1+events09:00:230
95078399697686,10cyclictest1203374-21kworker/0:1+events11:25:220
950793996960,694cyclictest1219940-21turbostat11:49:553
95078399696688,7cyclictest944104-21kworker/0:0+events07:20:440
950793996950,693cyclictest1032175-21turbostat08:19:553
95078399694686,6cyclictest1068581-21kworker/0:1+events09:26:490
95078399694685,8cyclictest925938-21kworker/0:2+events08:22:340
950793996930,691cyclictest1020030-21turbostat08:09:553
95078399693686,6cyclictest925938-21kworker/0:2+events08:00:410
95078399693685,6cyclictest925938-21kworker/0:2+events08:55:230
950793996900,688cyclictest950649-21turbostat07:14:563
950793996900,688cyclictest1128066-21turbostat09:34:563
95078399690683,6cyclictest925938-21kworker/0:2+events08:10:530
95078399688680,7cyclictest925938-21kworker/0:2+events08:49:450
95078399688678,9cyclictest925938-21kworker/0:2+events07:27:550
95078399687677,7cyclictest925938-21kworker/0:2+events07:48:170
95078399686679,6cyclictest925938-21kworker/0:2+events07:33:590
950793996830,681cyclictest1207470-21turbostat11:09:563
950793996810,679cyclictest1212136-21turbostat11:24:553
950793996800,678cyclictest1134986-21turbostat09:39:553
95079099659655,3cyclictest1235356-21turbostat12:39:552
95079399657654,1cyclictest0-21swapper/311:53:203
95079399655652,1cyclictest0-21swapper/312:11:163
95079399655652,1cyclictest0-21swapper/309:57:133
95079399653650,1cyclictest0-21swapper/312:33:283
95079399653650,1cyclictest0-21swapper/310:02:543
95079399652649,1cyclictest0-21swapper/312:22:443
95079399652649,1cyclictest0-21swapper/311:57:113
95079399652649,1cyclictest0-21swapper/310:56:153
95079399651648,1cyclictest0-21swapper/310:32:513
95079399651648,1cyclictest0-21swapper/309:42:203
95079399651648,1cyclictest0-21swapper/308:39:233
95079399651648,1cyclictest0-21swapper/307:35:423
95079399650647,1cyclictest0-21swapper/312:39:233
95079399650647,1cyclictest0-21swapper/312:16:303
95079399649646,1cyclictest0-21swapper/310:27:103
95079399649646,1cyclictest0-21swapper/309:06:173
95079399647644,1cyclictest0-21swapper/307:20:443
95079399646643,1cyclictest0-21swapper/312:02:313
95079399646643,1cyclictest0-21swapper/307:59:453
95079399645642,1cyclictest0-21swapper/310:53:233
95079399645642,1cyclictest0-21swapper/308:00:413
95079399644642,1cyclictest0-21swapper/311:25:223
95079399644641,1cyclictest0-21swapper/309:26:493
95079399644641,1cyclictest0-21swapper/309:16:593
95079399644641,1cyclictest0-21swapper/308:55:233
95079399644641,1cyclictest0-21swapper/308:22:343
95079399644641,1cyclictest0-21swapper/307:51:283
95079399643640,1cyclictest0-21swapper/310:41:303
95079399643640,1cyclictest0-21swapper/310:21:073
95079399643640,1cyclictest0-21swapper/309:47:303
95079399643640,1cyclictest0-21swapper/309:03:163
95079399642639,1cyclictest0-21swapper/307:18:373
95079399640637,1cyclictest0-21swapper/311:35:433
95079399640637,1cyclictest0-21swapper/310:37:103
95079399639636,1cyclictest0-21swapper/311:31:063
95079399639636,1cyclictest0-21swapper/311:12:543
95079399639636,1cyclictest0-21swapper/308:49:453
95079399638635,1cyclictest0-21swapper/311:18:283
95079399638635,1cyclictest0-21swapper/311:01:063
95079399638635,1cyclictest0-21swapper/308:41:053
95079399638635,1cyclictest0-21swapper/307:33:593
95079399638635,1cyclictest0-21swapper/307:29:233
95079399633630,1cyclictest0-21swapper/307:40:433
950790995134,8cyclictest1195000-21systemd10:27:032
950788993215,15cyclictest1194999-21systemd-user-ru10:27:031
950788992623,2cyclictest1224509-21sh12:00:001
95078899260,24cyclictest1134233-21anacron09:31:581
950790992520,3cyclictest522-21avahi-daemon12:15:012
950788992119,1cyclictest0-21swapper/111:10:001
950788992117,3cyclictest543-21gdbus11:25:001
950788992117,2cyclictest0-21swapper/110:25:001
950788992115,4cyclictest1170445-21grep10:00:181
950790992017,2cyclictest522-21avahi-daemon12:20:012
950788992018,1cyclictest0-21swapper/107:30:011
950788992017,2cyclictest1142063-21sendmail09:40:001
121983422016,2sleep30-21swapper/311:44:413
950790991916,2cyclictest0-21swapper/207:15:012
950790991915,3cyclictest1192729-21cut10:20:012
950790991915,2cyclictest0-21swapper/207:25:002
950790991815,2cyclictest533-21systemd-logind10:10:002
950788991815,2cyclictest1032285-21latency_hist08:15:001
950788991815,2cyclictest0-21swapper/108:40:001
291998180,3rtkit-daemon747-21snmpd10:06:221
950790991715,1cyclictest0-21swapper/208:05:002
950790991714,2cyclictest1220050-21latency_hist11:45:002
950788991714,2cyclictest1203755-21idleruntime-cro10:55:011
950788991713,3cyclictest1098729-21ntp_kernel_err09:05:231
950783991714,2cyclictest0-21swapper/011:45:000
12320412170,1sleep10-21swapper/112:24:351
950790991613,2cyclictest529-21polkitd08:50:012
950790991613,2cyclictest522-21avahi-daemon09:15:242
950790991613,2cyclictest1169887-21idleruntime-cro10:00:002
950790991613,2cyclictest1135812-21irqcore09:35:272
950790991613,2cyclictest1087507-21sendmail09:00:002
950790991613,2cyclictest0-21swapper/211:20:002
950790991612,3cyclictest361-21systemd-journal11:39:592
950790991612,3cyclictest1121686-21gltestperf09:25:482
950790991611,4cyclictest1119776-21grep09:20:432
95079099160,4cyclictest1028988-21grep08:10:152
950788991613,2cyclictest543-21gdbus09:45:551
950788991613,2cyclictest1216796-21latency_hist11:35:001
950788991613,2cyclictest1092618-21needreboot09:00:231
950783991612,2cyclictest0-21swapper/011:50:000
95079099154,1cyclictest471rcuog/209:31:342
950790991513,1cyclictest0-21swapper/210:25:002
950790991512,2cyclictest1032901-21irqcore08:15:142
950790991512,2cyclictest0-21swapper/210:40:012
950790991512,2cyclictest0-21swapper/208:00:002
950790991511,3cyclictest1068721-21unixbench-2d08:40:212
950790991511,2cyclictest1203986-21latency_hist10:55:012
950788991512,2cyclictest989626-21sendmail07:40:011
950788991512,2cyclictest959389-21fschecks_time07:15:111
950788991512,2cyclictest1159126-21kthreadcore09:50:221
950788991512,2cyclictest1142835-21tr09:40:171
950788991512,2cyclictest0-21swapper/108:20:001
950788991511,3cyclictest1045447-21gltestperf08:25:121
95078899150,14cyclictest0-21swapper/107:55:101
95079099141,8cyclictest1207471-21munin-run11:05:012
950790991411,2cyclictest1-21systemd09:05:252
950790991411,2cyclictest1043089-21kthreadcore08:20:152
950790991411,2cyclictest0-21swapper/210:45:012
950790991410,3cyclictest784-21gmain12:00:002
950788991412,1cyclictest956893-21latency_hist07:15:001
950788991412,1cyclictest1044712-21apt-key08:25:001
95078899141,1cyclictest1081665-21apt-get08:55:131
950788991411,2cyclictest1126249-21missed_timers09:25:551
950788991411,2cyclictest1106955-21latency_hist09:15:011
950788991411,2cyclictest1080496-21ntp_kernel_err08:50:301
950788991411,2cyclictest0-21swapper/112:40:011
950788991410,3cyclictest965679-21tail07:20:131
950788991410,2cyclictest0-21swapper/110:35:011
95078899140,2cyclictest10738612cut08:45:311
95078899140,1cyclictest0-21swapper/110:10:151
950783991411,1cyclictest0-21swapper/009:50:150
95079099139,3cyclictest977900-21fschecks_time07:30:122
95079099139,2cyclictest0-21swapper/212:10:012
95079099139,2cyclictest0-21swapper/209:00:122
95079099138,4cyclictest803-21gdbus10:59:232
95079099137,5cyclictest0-21swapper/207:35:132
95079099137,4cyclictest1080537-21ntpq08:50:322
950790991310,2cyclictest784-21gmain11:50:002
950790991310,2cyclictest784-21gmain08:30:012
950790991310,2cyclictest540-21in:imuxsock08:10:002
950790991310,2cyclictest1105445-21unixbench-2d09:10:392
950790991310,2cyclictest1063162-21sh08:40:002
950790991310,2cyclictest1006962-21ntp_kernel_pll_07:50:172
950790991310,2cyclictest0-21swapper/212:35:002
950790991310,2cyclictest0-21swapper/210:30:012
950790991310,2cyclictest0-21swapper/207:45:162
95078899139,3cyclictest759-21sshd08:32:321
95078899138,4cyclictest361-21systemd-journal11:05:011
950788991310,2cyclictest994261-21kthreadcore07:40:171
950788991310,2cyclictest533-21systemd-logind11:00:011
950788991310,2cyclictest1099632-21diskmemload12:30:001
950788991310,2cyclictest1022737-21kthreadcore08:05:151
950788991310,2cyclictest0-21swapper/108:40:101
950788991310,2cyclictest0-21swapper/107:50:121
950788991310,1cyclictest0-21swapper/107:45:211
95078899130,2cyclictest1211643-21ssh11:18:201
95079099129,2cyclictest994889-21proc_pri07:40:222
95079099129,2cyclictest1144336-21kthreadcore09:40:212
95079099129,2cyclictest1050141-21proc_pri08:25:202
95079099129,2cyclictest0-21swapper/210:00:142
950790991210,1cyclictest1099632-21diskmemload11:22:502
950790991210,1cyclictest0-21swapper/212:30:002
950790991210,1cyclictest0-21swapper/211:26:012
950790991210,1cyclictest0-21swapper/210:10:262
95079099120,12cyclictest0-21swapper/211:07:482
95079099120,11cyclictest0-21swapper/211:30:512
95078899129,2cyclictest1163609-21gltestperf09:55:161
95078899129,2cyclictest0-21swapper/108:05:001
95078899129,1cyclictest0-21swapper/110:45:011
95078899126,4cyclictest1107995-21sed09:15:151
950788991210,1cyclictest0-21swapper/112:30:331
950788991210,1cyclictest0-21swapper/110:47:341
95078899120,1cyclictest0-21swapper/112:08:091
95078899120,1cyclictest0-21swapper/110:17:141
95078899120,11cyclictest0-21swapper/112:03:551
95078899120,11cyclictest0-21swapper/111:50:271
95078899120,11cyclictest0-21swapper/111:28:171
95078899120,11cyclictest0-21swapper/111:10:251
95078899120,11cyclictest0-21swapper/109:20:161
95078899120,10cyclictest0-21swapper/112:10:091
291998120,3rtkit-daemon534-21wpa_supplicant11:50:092
95079099118,2cyclictest0-21swapper/207:10:012
95079099116,4cyclictest975384-21tr07:25:172
95078899119,1cyclictest0-21swapper/112:15:211
95078899119,1cyclictest0-21swapper/107:30:301
95078899110,11cyclictest0-21swapper/110:37:071
95079099109,1cyclictest0-21swapper/212:24:582
95079099109,1cyclictest0-21swapper/211:11:392
95079099106,4cyclictest950778-21cyclictest12:04:322
95079099105,4cyclictest961288-21cut07:15:142
9491612101,6sleep30-21swapper/307:05:173
9448582100,4sleep0944891-21apt-get07:05:100
9507909998,1cyclictest0-21swapper/210:47:462
944831270,2sleep1944353-21apt-get07:05:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional