You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 03:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Dec 09, 2025 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3999991313,34cyclictest11084-21apt-get22:58:342
3999991313,34cyclictest11084-21apt-get22:58:342
297121309,91sleep22496-21/usr/sbin/munin19:03:502
3285212110,79sleep13374-21/usr/sbin/munin19:04:021
288721097,79sleep00-21swapper/019:03:480
83121060,10sleep2371rcuc/220:34:552
83121060,10sleep2371rcuc/220:34:552
3992991022,71cyclictest2505-21ssh22:43:320
39999910156,38cyclictest355-21runrttasks21:48:332
39999910156,38cyclictest355-21runrttasks21:48:332
3999991013,29cyclictest19153-21ssh22:16:552
399299976,79cyclictest23669-21ssh23:18:500
399299952,83cyclictest26141-21grep22:28:540
399999942,77cyclictest21063-21kworker/u19:2+rpciod23:23:362
399799948,62cyclictest311-21apt-get23:33:351
3992999465,20cyclictest20606-21apt-key19:58:190
3992999465,20cyclictest20606-21apt-key19:58:190
3992999463,22cyclictest691-21cron21:03:160
3999999377,9cyclictest0-21swapper/223:58:202
399799934,36cyclictest29053-21/usr/sbin/munin20:23:391
399799933,41cyclictest31727-21apt-get00:28:261
399799933,32cyclictest21029-21proc_pri23:13:501
3997999331,10cyclictest0-21swapper/122:49:471
3992999312,48cyclictest15-21ksoftirqd/000:16:380
3997999236,51cyclictest413-21kworker/u17:2+events_unbound19:18:161
3997999228,13cyclictest0-21swapper/123:19:121
3997999226,12cyclictest0-21swapper/122:43:201
3997999218,32cyclictest10578-21cat21:03:521
399299923,75cyclictest23581-21chrt21:29:160
399799913,42cyclictest3396-21mailstats22:44:211
399799913,35cyclictest20632-21latency_hist19:58:181
399799913,35cyclictest20632-21latency_hist19:58:181
3997999131,11cyclictest0-21swapper/119:38:501
399299916,66cyclictest21063-21kworker/u19:2+ext4-rsv-conversion23:24:020
399299913,79cyclictest5607-21apt-get22:48:360
3992999113,49cyclictest15-21ksoftirqd/019:10:510
3992999113,49cyclictest15-21ksoftirqd/019:10:510
3999999046,36cyclictest0-21swapper/220:13:162
399799903,36cyclictest31157-21sshd22:37:241
399799903,33cyclictest7431-21grep20:53:561
399799903,30cyclictest21096-21timerandwakeup21:23:521
3992999019,43cyclictest15-21ksoftirqd/019:50:550
399999892,41cyclictest28786-21latency_hist21:38:182
399999892,23cyclictest304692sh22:35:222
399799894,30cyclictest26468-21/usr/sbin/munin00:18:401
399799894,30cyclictest26468-21/usr/sbin/munin00:18:401
399799893,39cyclictest17557-21ssh00:03:161
399799893,39cyclictest12728-21sort19:33:331
399799893,36cyclictest20957-21fschecks_count00:08:391
399799893,36cyclictest20957-21fschecks_count00:08:391
3997998933,48cyclictest0-21swapper/121:53:501
3997998932,7cyclictest0-21swapper/121:42:531
3997998931,49cyclictest0-21swapper/122:38:431
399799892,34cyclictest23721-21switchtime22:23:521
3997998921,12cyclictest0-21swapper/100:38:161
399999883,41cyclictest7278-21ssh23:43:542
399799883,37cyclictest16731-21latency_hist22:13:191
399799883,33cyclictest2608-21grep20:38:521
399799883,33cyclictest2608-21grep20:38:521
399799882,35cyclictest9334-21sort22:53:531
399299884,60cyclictest14594-21latency_hist23:58:190
399999873,74cyclictest23461-21ssh21:28:532
399799874,35cyclictest9074-21grep20:58:541
399799874,33cyclictest14471-21cstates21:13:351
399799873,37cyclictest17327-21fschecks_time22:13:391
399799873,35cyclictest23374-21timerandwakeup20:03:531
399799873,30cyclictest14999-21ssh23:03:431
399799873,27cyclictest12147-21grep22:03:521
3997998727,12cyclictest0-21swapper/122:59:071
3997998727,12cyclictest0-21swapper/122:59:071
399799872,34cyclictest355-21runrttasks23:08:451
399799872,34cyclictest18239-21grep19:48:501
399299876,50cyclictest15-21ksoftirqd/000:23:510
399299876,50cyclictest15-21ksoftirqd/000:23:510
399299873,50cyclictest15-21ksoftirqd/023:38:190
3992998716,34cyclictest15-21ksoftirqd/019:47:230
3999998644,35cyclictest0-21swapper/219:38:542
399999863,36cyclictest22280-21runrttasks22:22:292
399999863,35cyclictest3537-21/usr/sbin/munin00:33:522
399999863,34cyclictest2093-21sort20:38:412
399999863,34cyclictest2093-21sort20:38:412
399999862,40cyclictest22876-21ssh22:23:282
399999862,33cyclictest10569-21seq19:25:332
399999862,32cyclictest6386-21sed21:53:522
399799864,29cyclictest20237-21fschecks_time22:18:381
399799863,35cyclictest32502-21/usr/sbin/munin20:34:431
399799863,35cyclictest32502-21/usr/sbin/munin20:34:431
3997998628,12cyclictest0-21swapper/100:23:341
3997998628,12cyclictest0-21swapper/100:23:341
399799862,35cyclictest20624-21latency_hist00:08:191
399299867,71cyclictest11802-21rm22:58:540
399299867,71cyclictest11802-21rm22:58:540
3992998663,18cyclictest17367-21kworker/u19:2+events_unbound21:48:160
399999855,36cyclictest20041-21modprobe23:13:152
399999854,9cyclictest2476-21apt-get22:43:582
399999853,75cyclictest12128-21sed22:03:512
399999853,39cyclictest26554-21cpu00:18:322
399999853,39cyclictest26554-21cpu00:18:322
399999853,36cyclictest6900-21/usr/sbin/munin19:13:532
399999853,36cyclictest11127-21diskmemload23:33:282
399999852,34cyclictest21881-21ssh23:15:082
399799854,33cyclictest2945-21/usr/sbin/munin21:48:511
399799854,33cyclictest2945-21/usr/sbin/munin21:48:511
399799854,31cyclictest17284-21/usr/sbin/munin21:18:561
399799854,31cyclictest17284-21/usr/sbin/munin21:18:561
399799853,36cyclictest355-21runrttasks23:41:561
399799853,36cyclictest355-21runrttasks23:41:561
399799853,34cyclictest5762-21munin-node21:58:261
399799853,34cyclictest5762-21munin-node21:58:261
399799853,32cyclictest6167-21/usr/sbin/munin23:43:421
399799853,31cyclictest355-21runrttasks20:16:221
399799853,29cyclictest32612-21latency_hist23:33:191
399799853,29cyclictest32612-21latency_hist23:33:191
399799853,29cyclictest23862-21ssh21:29:351
399799853,29cyclictest23718-21sort00:13:351
399799852,35cyclictest12124-21ssh23:53:341
399799852,34cyclictest4947-21timerandwakeup19:08:541
399799852,34cyclictest4947-21timerandwakeup19:08:541
399799852,34cyclictest22333-21modprobe20:03:161
3992998512,35cyclictest15-21ksoftirqd/021:19:380
3992998512,35cyclictest15-21ksoftirqd/021:19:380
3999998442,35cyclictest0-21swapper/223:39:532
3999998442,35cyclictest0-21swapper/223:39:532
399999843,33cyclictest32629-21/usr/sbin/munin22:38:512
399999843,33cyclictest32629-21/usr/sbin/munin22:38:512
399999843,29cyclictest12456-21date19:33:182
399999842,38cyclictest7888-21cat20:58:152
399999842,35cyclictest5632-21cat19:13:182
399999842,35cyclictest5632-21cat19:13:182
399799843,34cyclictest25439-21apt-get22:28:281
399799843,29cyclictest28100-21mailstats20:18:491
399799843,28cyclictest676-21taskset21:43:551
399799842,36cyclictest25731-21latency_hist20:13:191
3997998422,9cyclictest0-21swapper/119:28:341
399799842,27cyclictest30878-21fschecks_count20:28:381
399299846,49cyclictest15-21ksoftirqd/020:36:120
399299846,49cyclictest15-21ksoftirqd/020:36:120
399299845,21cyclictest15-21ksoftirqd/019:59:320
399299844,48cyclictest15-21ksoftirqd/022:33:310
399299843,55cyclictest15-21ksoftirqd/022:18:330
3999998360,16cyclictest32474-21latency_hist20:33:202
3999998353,23cyclictest26204-21latency20:13:422
399999834,35cyclictest10049-21taskset21:03:402
399999834,31cyclictest857-21processes00:29:582
399999833,39cyclictest316-21rs:main2
399999833,33cyclictest11449-21/usr/sbin/munin21:08:482
399999833,30cyclictest16930-21ssh21:18:152
399999833,27cyclictest355-21runrttasks23:04:162
399999832,36cyclictest5607-21apt-get22:48:482
399999832,35cyclictest1686-21ssh21:46:532
399799835,30cyclictest26194-21/usr/sbin/munin21:33:561
3997998336,7cyclictest0-21swapper/123:48:531
399799833,36cyclictest16163-21cut19:43:381
399799833,32cyclictest14262-21latency_hist21:13:201
399799833,28cyclictest10060-21processes19:23:481
399799832,27cyclictest4284-21grep20:43:541
399799832,27cyclictest4284-21grep20:43:541
399299833,46cyclictest15-21ksoftirqd/020:03:460
3992998314,30cyclictest15-21ksoftirqd/019:20:060
3999998262,15cyclictest23078-21kworker/u17:3+events_unbound00:18:152
399999824,33cyclictest7931-21/usr/sbin/munin19:18:332
399999823,34cyclictest17816-21ssh21:18:452
399999823,34cyclictest17816-21ssh21:18:452
399999823,33cyclictest9070-21unixbench_multi20:58:542
399999823,28cyclictest355-21runrttasks20:23:212
399999822,33cyclictest32694-21apt-key23:33:202
399999822,33cyclictest27867-21latency20:18:422
399999822,24cyclictest17513-21apt-get19:48:272
399799823,30cyclictest8791-21grep19:18:521
399299829,40cyclictest15-21ksoftirqd/020:48:380
399299829,40cyclictest15-21ksoftirqd/020:48:380
3992998256,16cyclictest11074-21ssh22:03:190
3992998256,16cyclictest11074-21ssh22:03:190
3992998246,29cyclictest0-21swapper/022:28:170
3992998243,19cyclictest181rcu_preempt23:33:010
3992998243,19cyclictest181rcu_preempt23:33:010
399299823,46cyclictest15-21ksoftirqd/020:57:530
399299822,51cyclictest11134-21modprobe21:08:160
3999998142,32cyclictest0-21swapper/223:58:452
399999813,37cyclictest15905-21taskset19:43:272
399999813,36cyclictest9876-21/usr/sbin/munin23:48:522
399999813,36cyclictest23701-21ssh23:18:542
399999813,25cyclictest29636-21apt-get23:28:322
399999812,36cyclictest20520-21fschecks_time21:23:392
399299814,44cyclictest15-21ksoftirqd/021:13:140
399299813,45cyclictest15-21ksoftirqd/020:22:080
399299812,49cyclictest15-21ksoftirqd/020:15:120
399299812,34cyclictest9538-21/usr/sbin/munin23:48:430
399999803,34cyclictest29272-21expr21:38:342
399999803,34cyclictest18697-21/usr/sbin/munin00:03:522
399999803,32cyclictest3046-21sh20:43:192
399999803,32cyclictest3046-21sh20:43:192
399999802,35cyclictest30220-21/usr/sbin/munin00:23:552
399999802,35cyclictest30220-21/usr/sbin/munin00:23:552
399999802,33cyclictest23980-21/usr/sbin/munin20:08:172
3997998060,11cyclictest3978-21cyclictest20:53:191
3997998060,11cyclictest3978-21cyclictest20:53:191
3992998042,15cyclictest181rcu_preempt19:23:270
399299803,50cyclictest15-21ksoftirqd/020:10:590
399299803,34cyclictest15-21ksoftirqd/020:32:220
399299802,46cyclictest15-21ksoftirqd/019:15:100
3992998014,41cyclictest15-21ksoftirqd/000:01:170
3992998011,35cyclictest15-21ksoftirqd/020:28:160
399999793,35cyclictest13562-21grep19:33:522
399999793,34cyclictest21436-21mailstats00:08:512
399999793,34cyclictest21436-21mailstats00:08:512
399999793,34cyclictest20446-21cron19:58:162
399999793,34cyclictest20446-21cron19:58:162
399999793,34cyclictest15065-21/usr/sbin/munin22:08:532
399999792,34cyclictest21296-21fschecks_time19:59:182
399999792,31cyclictest5148-21fschecks_time20:48:382
399999792,31cyclictest5148-21fschecks_time20:48:382
399999792,29cyclictest26330-21fschecks_count22:29:002
399799793,31cyclictest26929-21grep23:24:081
399299793,42cyclictest6777-21/usr/sbin/munin23:43:450
399299792,48cyclictest15-21ksoftirqd/000:34:330
399299792,44cyclictest15-21ksoftirqd/023:07:560
399999783,32cyclictest8313-21/usr/sbin/munin21:58:502
399999783,32cyclictest8313-21/usr/sbin/munin21:58:502
399299789,45cyclictest15-21ksoftirqd/022:07:070
399299783,42cyclictest15-21ksoftirqd/021:39:140
399299783,39cyclictest15-21ksoftirqd/022:54:270
3992997829,22cyclictest15-21ksoftirqd/021:55:210
3992997762,9cyclictest21025-21sh21:23:510
3992997755,15cyclictest14546-21fschecks_time19:38:390
399299775,42cyclictest355-21runrttasks00:21:130
399299775,42cyclictest355-21runrttasks00:21:130
3992997753,15cyclictest11127-21diskmemload21:13:430
399299774,38cyclictest355-21runrttasks22:40:200
399299774,38cyclictest355-21runrttasks22:40:200
399299774,36cyclictest15-21ksoftirqd/022:12:590
399299774,36cyclictest15-21ksoftirqd/000:03:320
399299774,35cyclictest15-21ksoftirqd/020:39:200
399299774,35cyclictest15-21ksoftirqd/020:39:200
3992997743,16cyclictest181rcu_preempt23:16:370
3992997739,27cyclictest0-21swapper/021:33:460
399299773,41cyclictest4072-21switchtime23:38:510
399299773,41cyclictest4072-21switchtime23:38:510
399299773,26cyclictest15-21ksoftirqd/023:10:450
3992997720,33cyclictest445-21fschecks_time00:29:470
399299765,37cyclictest15-21ksoftirqd/000:11:180
399299765,37cyclictest15-21ksoftirqd/000:11:180
3992997540,12cyclictest181rcu_preempt19:38:060
3992997529,27cyclictest15-21ksoftirqd/022:16:450
400699748,40cyclictest0-21swapper/321:28:423
399299743,34cyclictest15-21ksoftirqd/021:49:030
399299743,34cyclictest15-21ksoftirqd/021:49:030
399299703,56cyclictest11280-21grep19:28:390
286726910,38sleep30-21swapper/319:03:473
400699663,37cyclictest0-21swapper/323:24:013
399299663,40cyclictest15-21ksoftirqd/020:47:490
399299663,40cyclictest15-21ksoftirqd/020:47:490
400699642,27cyclictest0-21swapper/323:18:323
400699618,22cyclictest0-21swapper/300:28:463
400699617,44cyclictest20996-21chrt19:58:313
4006996012,32cyclictest0-21swapper/322:43:303
4006995912,33cyclictest0-21swapper/322:48:333
400699584,32cyclictest0-21swapper/322:28:553
400699574,39cyclictest14727-21taskset22:08:463
400699573,38cyclictest14610-21chrt23:03:333
4006995610,14cyclictest0-21swapper/323:34:323
400699553,35cyclictest9575-21chrt19:23:333
400699543,35cyclictest20653-21chrt00:08:203
400699543,35cyclictest20653-21chrt00:08:203
400699539,29cyclictest0-21swapper/322:58:523
400699539,29cyclictest0-21swapper/322:58:523
400699535,32cyclictest6405-21taskset20:53:183
400699535,32cyclictest6405-21taskset20:53:183
400699533,38cyclictest3246-21chrt23:38:263
400699533,38cyclictest3246-21chrt23:38:263
400699533,37cyclictest4689-21taskset20:48:163
400699533,37cyclictest4689-21taskset20:48:163
400699533,37cyclictest25994-21taskset20:13:383
400699533,35cyclictest14767-21chrt23:58:193
400699532,40cyclictest17916-21taskset22:13:523
400699523,38cyclictest14791-21taskset21:13:433
400699523,33cyclictest3170-21chrt20:43:183
400699523,33cyclictest3170-21chrt20:43:183
400699522,39cyclictest11715-21taskset21:08:413
400699514,32cyclictest17389-21chrt19:48:193
400699513,40cyclictest3823-21taskset00:34:193
400699513,37cyclictest11433-21chrt22:03:353
400699513,34cyclictest9118-21chrt23:48:313
400699513,32cyclictest28992-21chrt21:38:193
400699512,40cyclictest12434-21taskset19:33:173
400699512,37cyclictest31630-21taskset21:43:013
400699503,38cyclictest8894-21chrt23:48:193
400699503,35cyclictest22977-21chrt23:18:163
400699503,34cyclictest31299-21chrt00:26:563
400699503,34cyclictest31299-21chrt00:26:563
400699503,34cyclictest19672-21chrt00:06:093
400699503,32cyclictest24316-21chrt00:13:513
400699503,32cyclictest22907-21taskset20:03:413
400699502,38cyclictest24237-21chrt22:24:423
400699502,37cyclictest8653-21chrt22:53:363
400699502,35cyclictest4952-21taskset21:52:533
400699502,35cyclictest4952-21taskset21:52:533
400699493,34cyclictest32631-21chrt23:33:193
400699493,34cyclictest32631-21chrt23:33:193
400699493,33cyclictest5737-21chrt21:53:383
400699493,33cyclictest12846-21taskset19:33:373
400699492,34cyclictest30524-21chrt22:35:323
400699483,36cyclictest20461-21taskset22:18:423
400699483,34cyclictest32715-21chrt21:43:503
400699483,33cyclictest9626-21taskset21:03:183
400699483,33cyclictest768-21chrt22:39:503
400699483,33cyclictest768-21chrt22:39:503
400699483,30cyclictest5141-21chrt19:08:563
400699483,30cyclictest5141-21chrt19:08:563
400699482,34cyclictest31613-21taskset20:28:523
400699482,34cyclictest20690-21taskset19:58:193
400699482,34cyclictest20690-21taskset19:58:193
400699482,32cyclictest28363-21chrt20:18:523
400699482,31cyclictest20068-21chrt21:23:193
400699482,31cyclictest20068-21chrt21:23:193
400699482,31cyclictest10773-21chrt22:02:503
400699482,31cyclictest10773-21chrt22:02:503
400699472,35cyclictest9962-21chrt21:03:393
400699464,29cyclictest25586-21chrt20:13:173
400699463,32cyclictest392-21chrt20:34:443
400699463,32cyclictest392-21chrt20:34:443
400699462,31cyclictest19018-21chrt23:10:133
400699453,32cyclictest28407-21chrt00:21:333
400699453,32cyclictest28407-21chrt00:21:333
400699452,33cyclictest15289-21chrt23:58:403
400699443,33cyclictest8704-21taskset19:18:523
4006994310,19cyclictest0-21swapper/321:23:213
400699423,26cyclictest5986-21chrt19:13:253
400699413,27cyclictest14443-21taskset19:38:353
400699412,27cyclictest18398-21taskset19:48:513
400699392,27cyclictest7640-21chrt20:55:253
400699382,25cyclictest29080-21chrt20:23:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional