You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-17 - 08:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Mon Mar 17, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1043321500,1sleep120-21swapper/1223:47:114
1043321500,1sleep120-21swapper/1223:47:114
3228021340,0sleep1241ktimersoftd/100:18:331
1096821280,2sleep710969-21sshd21:43:0013
61822127103,19sleep130-21swapper/1319:14:265
62882119105,10sleep100-21swapper/1019:15:592
6177211995,19sleep80-21swapper/819:14:2114
6377211689,23sleep90-21swapper/919:16:3015
5364211399,9sleep10-21swapper/119:12:541
1552921080,1sleep70-21swapper/722:54:0813
2417221070,0sleep7721rcuc/723:49:1413
2417221070,0sleep7721rcuc/723:49:1413
6302210592,9sleep50-21swapper/519:16:1011
5374210572,28sleep110-21swapper/1119:13:043
3045421030,1sleep00-21swapper/000:34:510
2236621010,1sleep90-21swapper/919:33:2015
2524721000,2sleep14130-21ksoftirqd/1423:44:166
2524721000,2sleep14130-21ksoftirqd/1423:44:166
75282970,1sleep40-21swapper/423:41:4210
75282970,1sleep40-21swapper/423:41:4210
277562950,7sleep5685999cyclictest23:34:1311
277562950,7sleep5685999cyclictest23:34:1311
11832950,6sleep1685399cyclictest20:32:021
220222940,1sleep100-21swapper/1021:39:502
613929367,21sleep30-21swapper/319:14:009
218372930,1sleep90-21swapper/923:38:3415
218372930,1sleep90-21swapper/923:38:3415
105942920,0sleep140-21swapper/1421:22:156
621129175,6sleep60-21swapper/619:14:5112
197432910,1sleep30-21swapper/323:10:399
616228872,4sleep140-21swapper/1419:14:116
75082860,0sleep140-21swapper/1421:47:246
293128575,6sleep20-21swapper/219:12:078
61342840,1sleep140-21swapper/1423:08:106
203062840,2sleep9686499cyclictest21:39:3315
616928375,4sleep40-21swapper/419:14:1710
615528375,4sleep70-21swapper/719:14:0413
613028275,3sleep150-21swapper/1519:13:567
627328173,4sleep120-21swapper/1219:15:454
536328072,4sleep00-21swapper/019:12:530
250172800,1sleep233-21ksoftirqd/222:27:518
128982790,0sleep100-21swapper/1022:47:302
46412780,0sleep120-21swapper/1200:02:254
248742760,3sleep9686499cyclictest23:55:2515
248742760,3sleep9686499cyclictest23:55:2515
122462760,0sleep110-21swapper/1123:53:033
122462760,0sleep110-21swapper/1123:53:033
238132740,6sleep0685199cyclictest23:44:020
238132740,6sleep0685199cyclictest23:44:020
293462650,1sleep70-21swapper/722:07:2013
685499520,2cyclictest0-21swapper/221:50:158
61462520,0sleep50-21swapper/523:41:2711
61462520,0sleep50-21swapper/523:41:2711
685499510,3cyclictest32189-21env22:51:118
686799501,2cyclictest0-21swapper/1122:07:563
685499500,3cyclictest998-21dbus-daemon23:38:518
685499500,3cyclictest998-21dbus-daemon23:38:518
685499500,2cyclictest0-21swapper/223:32:038
685499500,2cyclictest0-21swapper/223:32:038
685399500,43cyclictest31044-21sshd00:01:501
208342500,1sleep60-21swapper/600:17:0212
166672500,1sleep70-21swapper/723:04:5013
685499490,2cyclictest1014-21systemd-logind23:15:518
685499490,1cyclictest0-21swapper/221:46:398
686799481,15cyclictest509-21systemd-journal22:06:003
685499480,2cyclictest26152-21sh00:11:438
685499480,2cyclictest0-21swapper/223:07:158
685499480,2cyclictest0-21swapper/221:57:118
685499480,2cyclictest0-21swapper/221:21:478
685499480,1cyclictest0-21swapper/221:40:518
685499480,1cyclictest0-21swapper/200:22:238
685399480,3cyclictest0-21swapper/100:40:151
685499470,1cyclictest0-21swapper/222:19:478
685499470,1cyclictest0-21swapper/221:30:478
685499470,1cyclictest0-21swapper/220:42:038
685499470,1cyclictest0-21swapper/220:27:038
685499470,1cyclictest0-21swapper/200:15:198
686799461,2cyclictest0-21swapper/1122:14:413
685499460,3cyclictest17377-21sshd00:04:398
685499460,3cyclictest1005-21NetworkManager22:03:478
685499460,2cyclictest0-21swapper/223:06:278
685499460,2cyclictest0-21swapper/222:41:078
685499460,2cyclictest0-21swapper/220:12:158
685499460,1cyclictest0-21swapper/223:51:558
685499460,1cyclictest0-21swapper/223:51:558
685499460,1cyclictest0-21swapper/222:44:278
685499460,1cyclictest0-21swapper/221:25:158
685499460,1cyclictest0-21swapper/220:22:278
685499460,1cyclictest0-21swapper/200:40:318
685499460,1cyclictest0-21swapper/200:01:598
685499450,2cyclictest998-21dbus-daemon00:44:238
685499450,2cyclictest21620-21sshd22:55:198
685499450,2cyclictest0-21swapper/223:30:398
685499450,2cyclictest0-21swapper/223:25:118
685499450,2cyclictest0-21swapper/222:24:318
685499450,2cyclictest0-21swapper/200:32:318
685499450,1cyclictest0-21swapper/223:53:198
685499450,1cyclictest0-21swapper/223:53:198
685499450,1cyclictest0-21swapper/222:57:118
685499450,1cyclictest0-21swapper/222:09:078
6853994510,32cyclictest29657-21sshd23:45:011
6853994510,32cyclictest29657-21sshd23:45:011
685399450,42cyclictest26086-21sshd23:22:311
6867994412,4cyclictest990-21sshd22:51:263
685499440,3cyclictest998-21dbus-daemon23:18:438
685499440,2cyclictest0-21swapper/200:28:038
685499440,1cyclictest0-21swapper/223:43:478
685499440,1cyclictest0-21swapper/223:43:478
685499440,1cyclictest0-21swapper/221:34:118
685499440,1cyclictest0-21swapper/220:48:238
685499440,1cyclictest0-21swapper/220:38:278
685499440,1cyclictest0-21swapper/219:27:038
685499440,1cyclictest0-21swapper/200:18:518
685399445,3cyclictest998-21dbus-daemon21:28:501
6853994411,30cyclictest12151-21sshd22:15:521
6853994410,31cyclictest19209-21sshd21:55:301
6871994320,20cyclictest2608-21sshd22:13:595
685499434,28cyclictest503850nfsd19:19:108
685499430,2cyclictest0-21swapper/221:52:198
685499430,1cyclictest0-21swapper/222:36:558
685499430,1cyclictest0-21swapper/222:14:358
685499430,1cyclictest0-21swapper/220:52:078
685499430,1cyclictest0-21swapper/220:07:038
685499430,1cyclictest0-21swapper/219:32:078
685399431,7cyclictest0-21swapper/122:07:411
309042430,1sleep70-21swapper/722:02:3113
685499420,1cyclictest0-21swapper/221:02:358
685499420,1cyclictest0-21swapper/221:02:358
685499420,1cyclictest0-21swapper/220:32:278
685499420,1cyclictest0-21swapper/220:02:278
685499420,1cyclictest0-21swapper/219:42:598
685399428,2cyclictest31848-21sshd21:36:311
6853994210,29cyclictest27219-21sh21:19:491
6853994210,29cyclictest27028-21sshd22:39:391
685399420,40cyclictest0-21swapper/123:07:401
685399420,39cyclictest31550-21sshd22:28:591
687199410,19cyclictest7049-21cups-browsed22:09:205
685499410,2cyclictest0-21swapper/221:07:038
685499410,2cyclictest0-21swapper/221:07:038
685499410,2cyclictest0-21swapper/219:37:198
685499410,2cyclictest0-21swapper/219:22:278
685499410,1cyclictest0-21swapper/220:17:278
685499410,1cyclictest0-21swapper/219:47:318
6853994111,28cyclictest10556-21sshd21:22:151
6863994019,3cyclictest7049-21cups-browsed22:43:3214
685499400,2cyclictest0-21swapper/220:57:318
685499400,2cyclictest0-21swapper/220:57:318
685499400,1cyclictest0-21swapper/221:16:598
685499400,1cyclictest0-21swapper/221:16:598
685399406,4cyclictest1030-21polkitd22:02:391
685399406,3cyclictest19983-21sshd00:43:271
685399406,3cyclictest1014-21systemd-logind22:56:231
685399405,3cyclictest998-21dbus-daemon00:22:161
685399404,3cyclictest998-21dbus-daemon00:07:141
6867993913,3cyclictest1033-21gdbus22:43:223
685499390,1cyclictest0-21swapper/219:57:278
685499390,1cyclictest0-21swapper/219:54:558
685399396,3cyclictest31803-2110-uname22:18:421
685399395,3cyclictest1076-21gdbus23:37:131
685399395,3cyclictest1076-21gdbus23:37:131
685399394,3cyclictest998-21dbus-daemon00:02:431
6853993912,24cyclictest15739-21sshd22:42:531
685399390,36cyclictest6794-21sshd23:14:181
685399390,36cyclictest2646-21sshd23:02:251
6871993817,3cyclictest8406-2110-uname21:17:135
6856993818,2cyclictest0-21swapper/322:43:439
685399387,3cyclictest15816-21sshd22:00:211
685399385,4cyclictest31904-21gdbus00:27:041
685399385,3cyclictest8213-21sshd23:30:311
685399384,4cyclictest1076-21gdbus23:20:261
685399384,2cyclictest0-21swapper/122:58:591
6853993810,25cyclictest15175-2110-uname22:47:541
685399380,8cyclictest8514-21sshd21:42:351
6851993817,19cyclictest0-21swapper/021:57:090
6872993732,3cyclictest23200-21sshd21:34:496
685399376,3cyclictest20617-21sshd21:39:371
685399371,6cyclictest998-21dbus-daemon23:55:521
685399371,6cyclictest998-21dbus-daemon23:55:521
685399370,34cyclictest1115-21sshd22:24:031
685399370,2cyclictest0-21swapper/123:34:181
685399370,2cyclictest0-21swapper/123:34:181
207822370,4sleep341-21ksoftirqd/323:27:149
77472360,0sleep110-21swapper/1121:26:453
6867993611,4cyclictest22173-2110-uname22:01:293
685399365,3cyclictest31518-21sshd00:35:031
685399365,3cyclictest22770-21timerwakeupswit19:33:211
6867993511,3cyclictest12478-21updatedb.mlocat00:15:583
6863993520,5cyclictest15426-21sshd22:47:5814
6856993522,2cyclictest93450irq/42-eth0-TxR22:06:489
685399357,3cyclictest15970-21cpuspeed_turbos19:27:131
685399350,33cyclictest6285-21sshd00:14:031
6851993519,2cyclictest5038-21nfsd22:43:270
6864993419,3cyclictest509-21systemd-journal22:43:1215
685399346,3cyclictest28062-21sshd23:50:001
685399346,3cyclictest28062-21sshd23:50:001
685399346,3cyclictest10018-21cpuspeed20:42:201
685399345,3cyclictest22567-21/usr/sbin/munin20:57:201
685399345,3cyclictest22567-21/usr/sbin/munin20:57:201
685399344,3cyclictest0-21swapper/122:34:051
685399344,2cyclictest0-21swapper/121:50:021
279422341,28sleep80-21swapper/823:39:3814
279422341,28sleep80-21swapper/823:39:3814
259402340,30sleep50-21swapper/523:44:2411
259402340,30sleep50-21swapper/523:44:2411
184482340,2sleep010-21ksoftirqd/021:44:130
8762330,30sleep00-21swapper/021:30:360
6872993329,3cyclictest0-21swapper/1423:27:576
6872993328,3cyclictest2624-2110-uname00:35:436
6872993327,3cyclictest93150irq/39-eth0-TxR21:52:316
685399335,3cyclictest11386-21proc_pri20:02:251
218762330,1sleep100-21swapper/1023:48:452
218762330,1sleep100-21swapper/1023:48:452
196802330,2sleep80-21swapper/822:54:5714
6872993227,3cyclictest15283-21sshd22:54:066
6872993227,3cyclictest0-21swapper/1422:35:496
6872993226,4cyclictest5902-21sshd22:14:376
685399324,3cyclictest12492-21cstates19:22:131
685399322,4cyclictest0-21swapper/119:37:171
95642310,1sleep80-21swapper/823:30:4914
82862310,2sleep120-21swapper/1221:42:344
6872993127,3cyclictest0-21swapper/1423:55:396
6872993127,3cyclictest0-21swapper/1423:55:396
6872993127,3cyclictest0-21swapper/1420:02:206
6872993127,3cyclictest0-21swapper/1400:07:536
6872993126,3cyclictest32068-21apt-get22:57:096
6872993126,3cyclictest0-21swapper/1421:44:586
6872993126,3cyclictest0-21swapper/1420:01:406
6872993125,4cyclictest14872-21sshd21:38:366
6867993112,3cyclictest1005-21NetworkManager23:16:333
6867993111,3cyclictest12908-21sshd22:53:383
685399317,3cyclictest4227-21kernelversion21:12:191
685399317,3cyclictest4227-21kernelversion21:12:191
685399317,3cyclictest20709-21switchtime20:53:191
685399310,3cyclictest9129-21needreboot19:17:211
6872993027,2cyclictest0-21swapper/1422:11:466
6872993026,3cyclictest0-21swapper/1423:37:456
6872993026,3cyclictest0-21swapper/1423:37:456
6872993026,3cyclictest0-21swapper/1423:26:336
6872993026,3cyclictest0-21swapper/1422:47:256
6872993026,3cyclictest0-21swapper/1422:02:256
6872993026,3cyclictest0-21swapper/1421:12:316
6872993026,3cyclictest0-21swapper/1421:12:316
6872993026,3cyclictest0-21swapper/1400:15:456
6872993026,2cyclictest0-21swapper/1421:19:026
6872993025,4cyclictest0-21swapper/1422:17:416
6872993025,3cyclictest27071-21sshd22:22:576
6872993025,3cyclictest19336-21sshd21:28:136
6872993024,4cyclictest471-21jbd2/md124p1-823:35:426
6872993024,4cyclictest471-21jbd2/md124p1-823:35:426
6872993024,4cyclictest4051-21sshd22:29:536
6872993024,4cyclictest22476-21uname22:44:036
6872993024,4cyclictest0-21swapper/1419:37:026
687199307,20cyclictest24053-21sshd22:06:485
685399307,3cyclictest14423-21cpuspeed_turbos20:47:131
685399300,9cyclictest3343-21sshd20:32:201
6873992918,2cyclictest5040-21nfsd22:43:107
6872992926,2cyclictest9880-21sshd23:58:106
6872992926,2cyclictest0-21swapper/1419:29:486
6872992925,3cyclictest0-21swapper/1421:58:486
6872992925,3cyclictest0-21swapper/1400:18:416
6872992924,3cyclictest0-21swapper/1423:12:506
6872992924,3cyclictest0-21swapper/1423:03:386
6872992923,4cyclictest28593-21sshd23:50:076
6872992923,4cyclictest28593-21sshd23:50:076
6872992923,4cyclictest0-21swapper/1400:26:406
686499298,3cyclictest891ktimersoftd/919:40:4515
686099297,15cyclictest641rcuc/600:29:3712
685399291,4cyclictest0-21swapper/120:23:541
685399291,3cyclictest0-21swapper/119:42:181
3532290,25sleep40-21swapper/400:40:3310
195642290,26sleep00-21swapper/023:00:110
179832290,25sleep130-21swapper/1323:26:535
111512290,4sleep990-21ksoftirqd/922:15:4015
6872992824,3cyclictest0-21swapper/1400:29:246
6872992823,4cyclictest0-21swapper/1423:17:586
6872992823,3cyclictest19469-21sshd00:38:156
6872992822,4cyclictest17000-21sshd22:37:486
6872992822,4cyclictest0-21swapper/1400:45:276
686499286,20cyclictest891ktimersoftd/900:25:0715
686399284,21cyclictest1033-21gdbus22:05:4314
686399283,3cyclictest31904-21gdbus22:10:2414
686099289,17cyclictest0-21swapper/622:06:3612
686099284,22cyclictest0-21swapper/623:37:4912
686099284,22cyclictest0-21swapper/623:37:4912
685699282,23cyclictest27108-21sshd22:07:109
6856992820,6cyclictest391rcuc/321:58:179
685399285,3cyclictest1599-21missed_timers19:47:221
685399284,2cyclictest13424-21/usr/sbin/munin20:07:251
39362280,2sleep80-21swapper/823:07:4914
319682280,25sleep40-21swapper/423:50:4810
319682280,25sleep40-21swapper/423:50:4810
227712280,1sleep60-21swapper/622:38:5312
150452280,0sleep130-21swapper/1300:21:105
72362270,2sleep57301-21sshd00:08:1211
6872992724,2cyclictest12807-21sshd00:03:466
6872992724,2cyclictest0-21swapper/1420:53:266
6872992724,2cyclictest0-21swapper/1420:32:356
6872992724,2cyclictest0-21swapper/1420:22:566
6872992723,3cyclictest0-21swapper/1420:57:276
6872992723,3cyclictest0-21swapper/1420:57:276
6872992723,3cyclictest0-21swapper/1420:10:176
6872992723,3cyclictest0-21swapper/1419:54:226
6872992723,3cyclictest0-21swapper/1419:42:356
6872992722,3cyclictest0-21swapper/1420:13:196
686799270,13cyclictest27583-21sshd23:17:293
686699271,22cyclictest1033-21gdbus22:13:242
686699271,20cyclictest0-21swapper/1022:09:122
686499274,20cyclictest891ktimersoftd/922:06:3715
6860992725,1cyclictest641rcuc/622:59:2412
686099270,25cyclictest0-21swapper/623:10:4012
685699271,24cyclictest5038-21nfsd23:02:229
685399276,3cyclictest6435-21if_eth019:57:151
685199274,20cyclictest0-21swapper/022:08:420
55952270,2sleep50-21swapper/500:36:1711
258472270,1sleep40-21swapper/421:50:2610
224262270,1sleep50-21swapper/500:38:4211
14582270,2sleep50-21swapper/523:29:1011
99002260,3sleep1210354-21df_abs22:47:174
6872992623,2cyclictest0-21swapper/1421:08:066
6872992623,2cyclictest0-21swapper/1421:08:066
6872992623,2cyclictest0-21swapper/1420:20:516
6872992623,2cyclictest0-21swapper/1419:17:566
6872992622,3cyclictest0-21swapper/1420:47:066
6872992622,3cyclictest0-21swapper/1420:38:096
6872992620,4cyclictest5040-21nfsd19:47:336
686699261,21cyclictest1033-21gdbus22:05:362
686499265,19cyclictest891ktimersoftd/900:28:3515
686499261,22cyclictest7049-21cups-browsed22:07:4715
686399261,22cyclictest26198-21sshd22:12:1714
686099266,15cyclictest0-21swapper/623:06:0212
686099263,21cyclictest5260-21sshd22:19:4012
686099263,20cyclictest0-21swapper/621:19:2212
686099262,22cyclictest0-21swapper/623:42:2512
686099262,22cyclictest0-21swapper/623:42:2512
686099261,23cyclictest0-21swapper/621:48:3412
6860992611,8cyclictest1293-21sshd22:07:5612
686099260,24cyclictest0-21swapper/621:45:3312
685699265,9cyclictest401ktimersoftd/323:55:449
685699265,9cyclictest401ktimersoftd/323:55:449
685699265,19cyclictest0-21swapper/300:02:579
685699264,19cyclictest998-21dbus-daemon00:23:429
685699261,22cyclictest1005-21NetworkManager22:12:559
6856992610,10cyclictest0-21swapper/322:54:239
685199261,22cyclictest7049-21cups-browsed22:12:060
685199261,22cyclictest7049-21cups-browsed22:05:450
47022260,1sleep100-21swapper/1023:24:252
321372260,0sleep50-21swapper/523:23:3211
263832260,22sleep110-21swapper/1121:50:343
141892260,23sleep130-21swapper/1323:47:295
141892260,23sleep130-21swapper/1323:47:295
121472261,3sleep120-21swapper/1221:54:044
118672260,0sleep150-21swapper/1500:37:157
687399256,17cyclictest0-21swapper/1521:37:207
687399251,22cyclictest1371ktimersoftd/1522:12:567
687399250,20cyclictest0-21swapper/1522:06:297
6872992522,2cyclictest0-21swapper/1420:42:056
6872992522,2cyclictest0-21swapper/1419:41:476
6872992521,3cyclictest0-21swapper/1419:22:106
6872992520,3cyclictest29994-21cpuspeed20:27:136
6866992519,3cyclictest18085-21run-parts22:43:192
686499259,14cyclictest11970-21nfsd422:47:2515
686499257,6cyclictest881rcuc/921:44:4915
686499256,17cyclictest0-21swapper/921:33:3615
6864992521,2cyclictest881rcuc/922:31:4115
6864992512,8cyclictest131rcu_preempt23:03:4415
686099259,9cyclictest0-21swapper/621:27:2312
686099257,16cyclictest0-21swapper/622:42:3812
686099256,17cyclictest0-21swapper/622:34:5512
686099255,18cyclictest0-21swapper/623:58:5412
686099254,19cyclictest509-21systemd-journal00:10:3312
686099253,20cyclictest19270-1kworker/6:1H23:12:1912
686099253,20cyclictest0-21swapper/621:36:4412
686099253,20cyclictest0-21swapper/600:24:4612
6860992517,6cyclictest22050irq/30-ahci[00023:54:0112
6860992517,6cyclictest22050irq/30-ahci[00023:54:0112
6860992513,10cyclictest22050irq/30-ahci[00000:46:5912
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional