You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 22:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Wed Nov 19, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2419021740,3sleep1668499cyclictest09:56:121
1883121620,3sleep3669699cyclictest09:46:113
1883121620,3sleep3669699cyclictest09:46:113
1773911060,4ptp4l401ktimersoftd/308:11:573
177391700,1ptp4l401ktimersoftd/310:57:083
102342660,2sleep30-21swapper/311:08:093
6679996512,29cyclictest31980-21expr09:06:090
6696996453,6cyclictest41-21ksoftirqd/313:41:133
6684996412,17cyclictest4576-21snmp_o5powerctl11:41:331
6684996410,18cyclictest12757-21sed12:31:291
6684996310,20cyclictest3567-21grep13:41:171
6684996210,17cyclictest25149-21sed10:46:171
6684996012,16cyclictest26971-21ssh11:30:231
6684996010,17cyclictest6096-21/usr/sbin/munin10:21:291
667999601,6cyclictest20124-21latency12:41:170
152912590,4sleep1668499cyclictest11:55:561
668499589,18cyclictest31767-21snmp_o5powerctl10:11:211
668499589,16cyclictest11776-21cat09:31:101
6684995810,12cyclictest20045-21expr11:21:121
667999581,3cyclictest31291-21seq10:55:010
6684995710,17cyclictest9100-21users11:46:311
668499568,18cyclictest30806-21ssh11:34:581
668499568,18cyclictest30806-21ssh11:34:581
668499568,18cyclictest25185-21ssh12:47:401
668499568,17cyclictest2125-21ssh13:00:351
668499568,15cyclictest22375-21ssh12:45:311
668499558,17cyclictest13888-21ssh10:32:051
668499555,16cyclictest0-21swapper/110:44:281
177391550,1ptp4l401ktimersoftd/311:17:563
44352544,10sleep20-21swapper/208:10:542
6684995310,15cyclictest15166-21snmp_o5powerctl09:36:191
177391530,1ptp4l401ktimersoftd/311:28:393
668499529,14cyclictest7307-21diskstats13:06:141
668499528,18cyclictest29574-21snmp_o5powerctl10:51:341
6684995210,15cyclictest26158-21hddtemp_smartct10:01:121
6696995039,6cyclictest41-21ksoftirqd/312:26:123
6684995010,12cyclictest1251-21cut10:16:151
667999508,25cyclictest27976-21cat13:31:190
668999494,6cyclictest321ktimersoftd/213:40:592
668999481,6cyclictest18258-21munin-run09:45:532
668999476,6cyclictest33-21ksoftirqd/210:38:312
668999472,6cyclictest321ktimersoftd/209:16:092
668999470,6cyclictest33-21ksoftirqd/209:11:102
6684994710,12cyclictest15026-21bc13:16:111
668999469,7cyclictest33-21ksoftirqd/213:07:112
668999465,8cyclictest33-21ksoftirqd/210:25:552
668999460,6cyclictest16895-21grep11:56:212
668499468,13cyclictest8674-21irqstats11:06:151
668499466,14cyclictest0-21swapper/111:36:101
668499466,14cyclictest0-21swapper/111:36:101
667999463,39cyclictest15682-21tr11:56:120
668999454,6cyclictest321ktimersoftd/211:00:552
6689994541,2cyclictest311rcuc/211:30:352
668999453,7cyclictest321ktimersoftd/211:41:352
668999453,6cyclictest321ktimersoftd/210:11:112
668999452,6cyclictest33-21ksoftirqd/212:05:572
668999452,6cyclictest321ktimersoftd/209:06:282
668999450,8cyclictest321ktimersoftd/212:51:212
667999453,36cyclictest0-21swapper/012:21:110
667999452,37cyclictest11042-21apt-get13:11:110
668999446,8cyclictest33-21ksoftirqd/213:41:272
668999446,7cyclictest33-21ksoftirqd/213:19:592
668999446,7cyclictest33-21ksoftirqd/208:56:222
668999444,8cyclictest321ktimersoftd/212:11:172
668999444,7cyclictest321ktimersoftd/210:26:102
668999444,6cyclictest33-21ksoftirqd/212:16:122
668999444,6cyclictest33-21ksoftirqd/210:46:102
668999442,8cyclictest321ktimersoftd/209:51:112
668999442,6cyclictest321ktimersoftd/210:01:152
668999441,5cyclictest321ktimersoftd/211:16:142
668999440,6cyclictest33-21ksoftirqd/213:31:322
6684994410,8cyclictest12007-21df11:11:111
667999442,32cyclictest3129-21apt-get11:41:130
667999440,2cyclictest32485-21diskmemload11:18:120
177391440,1ptp4l2487-21kworker/3:109:56:253
6689994338,3cyclictest311rcuc/213:14:452
668999433,6cyclictest24364-21/usr/sbin/munin12:46:262
668999433,5cyclictest33-21ksoftirqd/213:02:112
668999433,5cyclictest33-21ksoftirqd/212:01:102
668999433,5cyclictest33-21ksoftirqd/211:04:052
668999432,6cyclictest33-21ksoftirqd/210:55:312
6689994319,8cyclictest33-21ksoftirqd/211:13:042
668999430,8cyclictest321ktimersoftd/210:41:102
668999430,7cyclictest321ktimersoftd/211:51:162
6684994311,19cyclictest31901-21cpuspeed_turbos09:06:091
667999433,32cyclictest419-21apt-get10:16:080
177391430,1ptp4l8041-21kworker/3:008:45:523
6696994231,5cyclictest41-21ksoftirqd/313:16:113
668999426,8cyclictest33-21ksoftirqd/208:51:172
668999426,7cyclictest33-21ksoftirqd/212:32:202
668999426,7cyclictest33-21ksoftirqd/211:46:522
668999426,7cyclictest33-21ksoftirqd/210:31:142
668999425,8cyclictest33-21ksoftirqd/212:45:552
668999424,9cyclictest33-21ksoftirqd/211:32:352
668999424,9cyclictest33-21ksoftirqd/211:32:352
6689994238,2cyclictest311rcuc/211:06:262
668999423,6cyclictest33-21ksoftirqd/210:16:042
668999423,6cyclictest33-21ksoftirqd/208:36:072
668999422,7cyclictest321ktimersoftd/208:41:132
6689994222,9cyclictest33-21ksoftirqd/211:36:102
6689994222,9cyclictest33-21ksoftirqd/211:36:102
668999421,6cyclictest321ktimersoftd/212:36:162
668999420,5cyclictest321ktimersoftd/211:21:272
6684994214,15cyclictest19712-21ssh12:01:101
667999422,35cyclictest0-21swapper/012:01:100
667999422,34cyclictest23310-21apt-get09:56:050
667999422,34cyclictest0-21swapper/012:51:120
667999422,33cyclictest3172-21apt-get13:41:140
667999422,33cyclictest27876-21apt-get12:11:130
667999422,32cyclictest11195-21apt-get12:31:110
667999422,31cyclictest6483-21apt-get09:21:080
6696994128,10cyclictest41-21ksoftirqd/310:26:263
668999415,7cyclictest13825-21taskset08:26:212
668999414,7cyclictest33-21ksoftirqd/208:21:012
668999413,6cyclictest29432-21idleruntime09:01:102
668999412,6cyclictest321ktimersoftd/209:31:062
6689994122,6cyclictest33-21ksoftirqd/210:06:132
668999411,5cyclictest321ktimersoftd/212:21:152
668999410,6cyclictest19620-21ps13:21:192
668999410,6cyclictest19620-21ps13:21:192
668499417,19cyclictest0-21swapper/109:46:101
668499417,19cyclictest0-21swapper/109:46:101
668499415,15cyclictest26895-21ssh13:30:501
668499415,15cyclictest26895-21ssh13:30:501
667999413,32cyclictest27590-21apt-get11:31:110
667999413,32cyclictest27590-21apt-get11:31:110
667999413,32cyclictest17094-21apt-get08:36:060
667999413,32cyclictest0-21swapper/013:01:110
667999413,31cyclictest26572-21apt-get08:56:050
667999412,36cyclictest13166-21ptp4l10:31:170
667999412,33cyclictest8142-21apt-get10:26:080
177391410,1ptp4l401ktimersoftd/311:21:533
6696994030,7cyclictest41-21ksoftirqd/313:06:113
6696994028,3cyclictest41-21ksoftirqd/312:20:573
668999403,6cyclictest33-21ksoftirqd/213:26:372
668999403,6cyclictest33-21ksoftirqd/213:26:372
668999403,6cyclictest33-21ksoftirqd/212:28:392
6689994033,4cyclictest33-21ksoftirqd/209:35:572
668999401,6cyclictest321ktimersoftd/208:16:052
668499406,11cyclictest0-21swapper/112:10:591
6684994012,16cyclictest6218-21chrt13:05:341
667999404,30cyclictest0-21swapper/009:36:070
667999403,30cyclictest28938-21apt-get09:01:050
667999402,33cyclictest0-21swapper/011:21:080
667999402,32cyclictest20943-21apt-get09:51:060
667999402,30cyclictest7750-21apt-get11:06:080
667999401,5cyclictest32485-21diskmemload12:40:490
667999401,35cyclictest21979-21cpuspeed_turbos08:46:060
667999400,4cyclictest5283-21/usr/sbin/munin11:01:230
177391400,1ptp4l401ktimersoftd/311:39:423
177391400,1ptp4l401ktimersoftd/311:39:423
177391400,1ptp4l401ktimersoftd/310:32:523
669699396,4cyclictest41-21ksoftirqd/312:51:163
6696993933,3cyclictest41-21ksoftirqd/311:05:543
668499397,17cyclictest0-21swapper/109:21:201
668499396,20cyclictest0-21swapper/111:01:201
6684993911,11cyclictest30273-21snmp_o5powerctl09:01:181
6684993910,11cyclictest10219-21snmp_o5powerctl09:26:191
667999392,32cyclictest12358-21apt-get08:26:050
667999392,30cyclictest25676-21apt-get10:01:070
667999392,29cyclictest19457-21apt-get08:41:070
667999390,4cyclictest9070-21users12:26:330
667999390,4cyclictest23700-21cron10:45:550
6696993833,3cyclictest41-21ksoftirqd/313:25:583
6696993833,3cyclictest41-21ksoftirqd/313:25:583
668999384,7cyclictest33-21ksoftirqd/209:26:162
668999382,6cyclictest33-21ksoftirqd/212:57:492
668999380,5cyclictest33-21ksoftirqd/210:00:542
668499387,19cyclictest0-21swapper/110:26:201
668499387,18cyclictest0-21swapper/112:06:141
668499387,17cyclictest0-21swapper/108:16:151
668499384,10cyclictest0-21swapper/113:22:531
668499384,10cyclictest0-21swapper/113:22:531
6684993811,8cyclictest16200-21if_enp1s0.111:16:141
6684993811,12cyclictest30547-21awk12:55:571
667999383,30cyclictest15043-21apt-get13:16:120
667999383,29cyclictest32047-21apt-get10:56:090
667999382,31cyclictest31941-21apt-get12:16:110
667999382,31cyclictest22946-21apt-get12:46:100
667999382,30cyclictest32485-21diskmemload11:26:240
667999380,4cyclictest26161-21seq08:54:370
667999380,3cyclictest0-21swapper/012:06:290
177391380,1ptp4l401ktimersoftd/313:32:093
177391380,1ptp4l401ktimersoftd/313:29:453
177391380,1ptp4l401ktimersoftd/313:29:453
177391380,1ptp4l401ktimersoftd/313:01:063
177391380,1ptp4l401ktimersoftd/312:50:273
177391380,1ptp4l401ktimersoftd/311:58:243
6696993731,3cyclictest41-21ksoftirqd/313:40:593
668999372,5cyclictest33-21ksoftirqd/208:35:032
668499377,18cyclictest0-21swapper/112:36:101
6684993724,7cyclictest3474-21apt-get12:21:111
6684993710,12cyclictest16851-21sed10:36:161
667999377,26cyclictest10402-21gltestperf08:21:180
667999373,30cyclictest30426-21apt-get10:11:070
667999372,30cyclictest11245-21apt-get09:31:060
667999372,30cyclictest0-21swapper/009:11:050
667999370,4cyclictest4441-21fschecks_time09:16:090
667999370,4cyclictest11113-21date11:50:560
667999370,3cyclictest31570-21cstates13:36:150
177391370,1ptp4l401ktimersoftd/310:43:153
177391370,1ptp4l401ktimersoftd/310:16:453
669699368,3cyclictest41-21ksoftirqd/309:51:073
669699363,5cyclictest41-21ksoftirqd/309:35:533
6696993631,3cyclictest41-21ksoftirqd/311:35:553
6696993631,3cyclictest41-21ksoftirqd/311:35:553
6696993631,3cyclictest41-21ksoftirqd/310:50:553
6696993631,3cyclictest41-21ksoftirqd/309:16:133
6696993630,3cyclictest41-21ksoftirqd/313:15:583
6696993630,3cyclictest41-21ksoftirqd/308:45:523
6696993629,4cyclictest41-21ksoftirqd/312:06:153
6696993625,6cyclictest41-21ksoftirqd/309:02:343
6696993620,7cyclictest41-21ksoftirqd/312:41:193
6696993620,6cyclictest41-21ksoftirqd/309:26:233
668999361,7cyclictest121rcu_preempt09:46:172
668999361,7cyclictest121rcu_preempt09:46:172
668999361,6cyclictest321ktimersoftd/208:46:162
668499367,19cyclictest0-21swapper/109:51:061
668499365,18cyclictest0-21swapper/112:16:111
668499365,12cyclictest0-21swapper/113:31:561
667999362,29cyclictest15803-21apt-get10:36:080
667999360,4cyclictest20076-21cat09:46:250
667999360,4cyclictest20076-21cat09:46:250
177391360,1ptp4l401ktimersoftd/312:14:153
177391360,1ptp4l401ktimersoftd/311:48:203
177391360,0ptp4l401ktimersoftd/312:01:133
669699357,4cyclictest41-21ksoftirqd/309:06:123
6696993530,3cyclictest41-21ksoftirqd/312:56:143
6696993530,3cyclictest41-21ksoftirqd/309:45:533
6696993529,3cyclictest41-21ksoftirqd/312:36:163
6696993529,3cyclictest41-21ksoftirqd/308:16:083
6696993522,5cyclictest41-21ksoftirqd/308:31:063
668499359,14cyclictest19098-21ssh12:00:231
668499357,16cyclictest0-21swapper/108:36:061
668499356,18cyclictest0-21swapper/109:11:121
668499354,9cyclictest0-21swapper/110:57:081
6679993513,19cyclictest16444-21df_inode09:41:080
6679993510,22cyclictest7367-21sh13:06:150
667999350,4cyclictest9762-21irqstats09:26:130
667999350,4cyclictest7914-21memory08:16:110
667999350,2cyclictest0-21swapper/013:30:580
667999350,2cyclictest0-21swapper/013:30:580
667999350,2cyclictest0-21swapper/010:21:300
59692356,13sleep10-21swapper/108:11:251
49622356,9sleep00-21swapper/008:11:120
158642350,4sleep3669699cyclictest10:36:093
669699347,3cyclictest41-21ksoftirqd/310:11:083
669699346,4cyclictest41-21ksoftirqd/311:41:223
669699346,4cyclictest41-21ksoftirqd/308:36:223
669699346,3cyclictest41-21ksoftirqd/310:21:143
668999344,7cyclictest33-21ksoftirqd/209:25:522
6684993415,15cyclictest4567-21grep09:16:091
6684993410,9cyclictest26933-21perl08:56:081
667999348,22cyclictest28396-21fschecks_time10:06:130
667999340,4cyclictest27216-21ssh10:49:430
667999340,4cyclictest12292-21irqstats11:51:160
177391340,1ptp4l401ktimersoftd/312:21:113
177391340,1ptp4l401ktimersoftd/311:54:003
177391340,1ptp4l401ktimersoftd/309:21:143
669699335,4cyclictest41-21ksoftirqd/312:31:173
6696993328,3cyclictest41-21ksoftirqd/309:15:523
6696993328,3cyclictest41-21ksoftirqd/308:55:523
6696993327,3cyclictest41-21ksoftirqd/308:20:523
6696993316,7cyclictest41-21ksoftirqd/308:26:223
668499339,8cyclictest10239-21diskstats08:21:171
668499334,10cyclictest0-21swapper/113:36:441
668499334,10cyclictest0-21swapper/112:27:171
6684993315,12cyclictest0-21swapper/113:11:111
667999338,22cyclictest13507-21unixbench_singl11:11:290
667999337,23cyclictest19336-21fschecks_time13:21:170
667999337,23cyclictest19336-21fschecks_time13:21:170
667999334,26cyclictest32067-21/usr/sbin/munin12:56:220
667999330,30cyclictest16880-21cat08:35:520
177391330,1ptp4l401ktimersoftd/310:53:353
177391330,1ptp4l401ktimersoftd/309:39:303
669699326,8cyclictest41-21ksoftirqd/310:06:203
669699325,4cyclictest41-21ksoftirqd/308:56:103
6696993227,3cyclictest41-21ksoftirqd/311:11:163
668499328,10cyclictest0-21swapper/108:26:111
668499314,10cyclictest0-21swapper/108:50:361
667999311,6cyclictest1127-21ssh11:36:530
667999311,6cyclictest1127-21ssh11:36:530
669699304,3cyclictest41-21ksoftirqd/310:01:113
6684992916,9cyclictest17534-21snmp_o5powerctl09:41:201
668499287,8cyclictest24569-21perl08:51:081
668499277,7cyclictest20114-21ps08:41:131
6684992715,9cyclictest28715-21cut10:06:161
6684992715,9cyclictest15987-21sendmail_mailqu08:31:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional