You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 11:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Thu Sep 03, 2026 00:44:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295449912438,34cyclictest0-21swapper/022:50:120
295449911651,54cyclictest24079-21apt-get21:40:120
295449911637,30cyclictest0-21swapper/021:50:170
295679910839,12cyclictest0-21swapper/222:10:312
295679910741,11cyclictest0-21swapper/223:00:002
295679910536,14cyclictest0-21swapper/222:35:042
295679910536,14cyclictest0-21swapper/222:35:042
295449910545,20cyclictest8042-21apt-get21:15:450
295679910445,51cyclictest0-21swapper/221:40:382
295679910439,10cyclictest0-21swapper/219:50:042
295679910435,15cyclictest0-21swapper/223:14:432
295449910441,22cyclictest20664-21cat00:05:260
295449910436,27cyclictest22900-21ssh22:27:400
295449910239,53cyclictest0-21swapper/023:15:110
295449910239,53cyclictest0-21swapper/023:15:110
295679910172,22cyclictest200-21systemd-journal22:00:002
29567991013,39cyclictest23371-21head20:25:352
29567991013,38cyclictest26550-21latency_hist00:15:022
295679910131,21cyclictest31061-21cron19:15:012
295449910134,28cyclictest17557-21latency_hist21:30:040
295449910134,28cyclictest17557-21latency_hist21:30:040
29567991003,44cyclictest28065-21tr21:45:342
295449910039,19cyclictest5282-21grep23:40:370
29567999937,20cyclictest17495-21fschecks_time00:00:212
29567999937,11cyclictest0-21swapper/220:35:002
2955799993,77cyclictest20965-21apt-get22:25:161
2956799974,40cyclictest343-21runrttasks23:04:532
2956799974,40cyclictest343-21runrttasks23:04:532
2956799974,35cyclictest7650-21/usr/sbin/munin22:05:372
2956799973,77cyclictest17660-21ssh22:20:292
2956799973,29cyclictest14353-21uniq22:15:212
2955799974,59cyclictest10820-21apt-get23:50:161
29544999734,24cyclictest23926-21latency_hist20:30:030
29544999730,24cyclictest14099-21ssh21:24:560
2956799967,44cyclictest21128-21ssh22:25:352
29544999633,23cyclictest7863-21grep19:40:230
29567999543,11cyclictest29536-21cyclictest22:39:402
2956799953,41cyclictest20433-21acpi00:05:152
2956799953,30cyclictest17367-21latency_hist20:10:032
29544999535,21cyclictest14452-21cpuspeed20:00:400
29544999534,23cyclictest18332-21xargs20:10:340
29544999533,25cyclictest30428-21taskset23:30:210
29544999530,27cyclictest27423-21cstates20:40:180
29544999529,26cyclictest0-21swapper/020:55:150
29544999529,26cyclictest0-21swapper/020:55:150
2956799944,45cyclictest691-21latency_hist23:35:022
2955799943,74cyclictest2822-21modprobe19:25:151
29544999434,25cyclictest4269-21diskmemload21:30:340
29544999432,25cyclictest24570-21ntpq20:30:290
29544999431,20cyclictest18721-21kworker/u17:3+xprtiod00:14:310
2956799933,36cyclictest14251-21cmp20:00:042
29557999371,15cyclictest27714-21rm22:35:241
29544999342,36cyclictest0-21swapper/023:35:150
29544999328,29cyclictest19001-21ssh23:11:520
2956799924,39cyclictest742-21latency_hist19:20:032
29544999257,20cyclictest2918-21apt-get21:05:160
29544999235,42cyclictest10843-21ssh23:50:150
29544999232,24cyclictest19874-21swap20:15:340
29544999232,24cyclictest19874-21swap20:15:340
29544999229,29cyclictest16486-21ntpq20:05:330
2955799913,50cyclictest8042-21apt-get21:15:201
29544999135,14cyclictest21116-21tune2fs21:35:230
29544999133,22cyclictest17848-21ssh00:00:300
29544999133,18cyclictest9819-21awk19:45:340
29544999131,26cyclictest27038-21sh23:25:150
29544999127,25cyclictest0-21swapper/022:40:390
29567999048,11cyclictest181rcu_preempt23:35:132
29544999034,36cyclictest662-21cron23:50:000
29544999031,25cyclictest21716-21unixbench_multi20:20:360
2956799893,63cyclictest11059-21cstates23:50:392
29544998935,23cyclictest25765-21ssh22:31:520
29544998935,23cyclictest25765-21ssh22:31:520
29544998933,22cyclictest27603-21ssh00:15:350
29544998933,22cyclictest27603-21ssh00:15:350
29544998932,27cyclictest6975-21cut00:35:030
29544998932,27cyclictest6975-21cut00:35:030
29544998930,23cyclictest10742-21latency_hist23:00:030
29544998929,28cyclictest0-21swapper/000:35:380
29544998929,25cyclictest16564-21sh22:19:020
29544998924,28cyclictest0-21swapper/000:23:390
29567998838,9cyclictest0-21swapper/219:25:182
2956799883,37cyclictest29923-21ssh00:20:042
2956799883,37cyclictest29923-21ssh00:20:042
2956799883,36cyclictest1055-21grep00:25:222
29567998812,66cyclictest6387-21kworker/u18:1+flush-179:021:15:272
29557998848,14cyclictest25160-21ssh00:11:411
29544998835,46cyclictest4606-21taskset21:10:130
29544998835,46cyclictest4606-21taskset21:10:130
29544998835,38cyclictest343-21runrttasks00:27:590
29544998833,24cyclictest16967-21date00:00:030
29544998830,22cyclictest13788-21chrt19:59:440
29544998824,30cyclictest24121-21fschecks_time23:20:240
2956799871,41cyclictest8186-21grep19:40:292
29557998766,15cyclictest1-21systemd23:00:001
2955799873,64cyclictest16970-21apt-get22:20:161
29544998731,19cyclictest4668-21df_inode22:00:200
29544998731,19cyclictest4668-21df_inode22:00:200
29544998632,23cyclictest31514-21grep19:15:070
29544998632,20cyclictest17699-21chrt22:20:340
2956799859,27cyclictest14354-21apt-get20:00:182
2956799854,39cyclictest31614-21unixbench_multi22:40:382
29557998559,19cyclictest9356-21sh22:07:011
29544998535,38cyclictest30589-21/usr/sbin/munin20:50:170
29544998534,36cyclictest28902-21ssh21:46:530
29544998531,23cyclictest11248-21fschecks_time23:00:210
29544998531,23cyclictest11248-21fschecks_time23:00:210
29544998529,24cyclictest10555-21latency_hist22:10:030
29567998468,12cyclictest29536-21cyclictest20:25:032
2956799843,38cyclictest4359-21ssh22:50:032
2956799843,38cyclictest4359-21ssh22:50:032
29557998460,16cyclictest4269-21diskmemload21:36:401
29544998426,37cyclictest0-21swapper/022:10:100
29557998361,15cyclictest28791-21diskmemload21:46:361
29544998349,26cyclictest29536-21cyclictest19:10:020
29544998341,36cyclictest0-21swapper/022:35:160
29544998326,24cyclictest0-21swapper/019:25:470
29567998246,10cyclictest181rcu_preempt19:30:152
29557998254,19cyclictest18519-21sh00:01:031
29544998234,35cyclictest15382-21ssh23:05:490
29567998158,17cyclictest7733-21latency_hist21:15:042
29567998158,17cyclictest7733-21latency_hist21:15:042
29557998152,22cyclictest27470-21fschecks_count20:40:221
29557998148,7cyclictest181rcu_preempt23:44:001
29544998126,25cyclictest0-21swapper/021:55:230
29544998123,21cyclictest2247-21chrt19:24:250
29544998123,21cyclictest2247-21chrt19:24:250
29557998059,14cyclictest200-21systemd-journal22:32:401
29557998059,14cyclictest200-21systemd-journal22:32:401
29544998033,31cyclictest6603-21ps19:35:330
29544998022,28cyclictest2055-21processes22:45:330
29544998022,28cyclictest2055-21processes22:45:330
2955799799,59cyclictest14354-21apt-get20:00:341
29557997961,11cyclictest13749-21modprobe22:15:001
29557997956,15cyclictest30939-21timerandwakeup00:20:341
29544997945,24cyclictest25653-21apt-get20:35:120
29544997922,25cyclictest0-21swapper/019:10:210
29544997832,39cyclictest0-21swapper/019:55:040
29544997832,39cyclictest0-21swapper/019:55:040
29567997753,19cyclictest18330-21threads20:10:342
29544997742,25cyclictest28873-21/usr/sbin/munin20:45:260
29557997661,10cyclictest24891-21unixbench_singl23:20:391
29557997652,15cyclictest29924-21latency_hist00:20:041
29557997652,15cyclictest29924-21latency_hist00:20:041
29544997621,18cyclictest0-21swapper/019:30:120
29557997560,10cyclictest29536-21cyclictest23:55:171
29567997450,18cyclictest343-21runrttasks19:50:292
29567997450,18cyclictest343-21runrttasks19:50:292
29567997435,13cyclictest181rcu_preempt21:50:232
2956799743,47cyclictest30423-21latency_hist20:50:042
29557997459,8cyclictest0-21swapper/123:00:361
29557997459,8cyclictest0-21swapper/123:00:361
29557997457,13cyclictest29536-21cyclictest21:10:221
29557997457,13cyclictest29536-21cyclictest21:10:221
29557997451,16cyclictest29536-21cyclictest19:10:261
29557997451,11cyclictest30744-21kworker/u17:3+flush-179:019:15:051
2957399733,28cyclictest0-21swapper/321:15:413
29557997348,15cyclictest1003-21diskmemload21:55:021
29557997346,19cyclictest26955-21latency_hist21:45:041
2955799732,39cyclictest21039-21fschecks_count20:20:221
29567997242,21cyclictest1171-21fschecks_count19:20:232
29567997242,21cyclictest1171-21fschecks_count19:20:232
29567997241,12cyclictest29536-21cyclictest00:39:242
29567997232,10cyclictest181rcu_preempt20:35:142
29557997246,15cyclictest0-21swapper/120:55:201
29557997246,15cyclictest0-21swapper/120:55:201
29557997235,26cyclictest15217-21unixbench_multi23:05:381
29557997223,32cyclictest31-21ksoftirqd/121:56:351
29567997150,14cyclictest4269-21diskmemload23:55:182
29567997148,14cyclictest0-21swapper/200:25:022
2956799713,48cyclictest6890-21munin-run00:35:002
2956799713,48cyclictest6890-21munin-run00:35:002
29557997157,9cyclictest326-21systemd-logind20:50:021
29557997155,10cyclictest7458-21fschecks_count00:35:231
29557997139,12cyclictest181rcu_preempt19:45:041
29557997134,10cyclictest181rcu_preempt22:41:061
29567997049,14cyclictest21484-21/usr/sbin/munin21:35:312
2956799704,47cyclictest10447-21date23:50:032
29567997044,16cyclictest7357-21cron19:40:012
2956799702,45cyclictest24110-21fschecks_time23:20:242
29557997054,11cyclictest15027-21/usr/sbin/munin21:25:291
29557997054,11cyclictest15027-21/usr/sbin/munin21:25:291
29557997052,11cyclictest23471-21munin-run23:20:001
29557997052,11cyclictest23471-21munin-run23:20:001
29557997051,14cyclictest30347-21cstates23:30:181
29557997048,14cyclictest20630-21latency_hist21:35:031
29557997045,19cyclictest5171-21sh00:30:431
29557997045,19cyclictest5171-21sh00:30:431
29557997045,15cyclictest22642-21fschecks_count20:25:201
29557997045,14cyclictest0-21swapper/119:35:041
29557997033,25cyclictest18338-21wc23:10:411
29544997036,23cyclictest1460-21/usr/sbin/munin21:00:210
29567996952,12cyclictest13224-21runrttasks21:22:442
2956799693,46cyclictest27320-21fschecks_time23:25:232
29557996926,31cyclictest16674-21cat20:05:391
2955799692,36cyclictest10579-21latency_hist23:50:041
2956799687,48cyclictest4164-21/usr/sbin/munin23:40:222
29567996848,15cyclictest29536-21cyclictest22:55:042
2956799683,45cyclictest15274-21head23:05:392
29557996854,10cyclictest29536-21cyclictest22:51:511
29557996853,10cyclictest29536-21cyclictest22:04:001
29557996853,10cyclictest29536-21cyclictest22:04:001
2955799683,42cyclictest31443-21unixbench_singl20:50:371
2955799682,32cyclictest1101-21grep00:25:231
29557996816,31cyclictest31-21ksoftirqd/120:33:021
2956799673,51cyclictest17668-21ssh21:30:132
29567996729,28cyclictest26989-21kworker/u19:3+nfsiod22:00:392
29567996729,28cyclictest26989-21kworker/u19:3+nfsiod22:00:392
29557996748,9cyclictest0-21swapper/123:35:231
2955799672,33cyclictest12326-21ssh21:20:401
29567996630,13cyclictest181rcu_preempt20:59:172
29567996630,13cyclictest181rcu_preempt20:59:172
29567996624,13cyclictest181rcu_preempt21:05:262
29557996644,14cyclictest29536-21cyclictest20:40:031
29557996643,13cyclictest0-21swapper/119:25:031
29557996643,13cyclictest0-21swapper/119:25:031
29557996640,17cyclictest14301-21fschecks_count22:15:201
2956799655,39cyclictest343-21runrttasks23:19:132
2956799655,39cyclictest343-21runrttasks23:19:132
29567996539,14cyclictest0-21swapper/220:50:322
29557996548,11cyclictest22910-21sshd00:09:031
2955799653,31cyclictest13373-21unixbench_multi19:55:371
2955799653,13cyclictest181rcu_preempt20:20:031
2955799653,13cyclictest181rcu_preempt20:20:031
2956799644,38cyclictest27964-21sensors_temp20:40:342
2956799644,37cyclictest2655-21modprobe21:05:002
29557996440,15cyclictest10942-21cpuspeed19:50:171
29557996440,15cyclictest10942-21cpuspeed19:50:171
2955799643,25cyclictest17803-21sort20:10:211
2957399632,38cyclictest2824-21taskset19:25:173
2956799634,42cyclictest19167-21/usr/sbin/munin20:15:112
2956799634,42cyclictest19167-21/usr/sbin/munin20:15:112
2955799634,25cyclictest31-21ksoftirqd/122:47:441
2955799634,25cyclictest31-21ksoftirqd/122:47:441
2956799624,32cyclictest15079-21ntp_offset21:25:302
2956799624,32cyclictest15079-21ntp_offset21:25:302
29557996240,12cyclictest0-21swapper/123:25:351
2955799622,38cyclictest10049-21timerandwakeup19:45:381
2955799622,35cyclictest6527-21ntp_kernel_pll_19:35:291
2955799622,24cyclictest3142-21fschecks_time21:05:221
2955799613,37cyclictest2099-21ls21:00:371
2956799604,39cyclictest29628-21latency_hist19:10:032
29557995836,11cyclictest0-21swapper/119:10:031
2957399567,39cyclictest9672-21chrt00:39:003
2957399564,40cyclictest21293-21taskset21:35:273
2957399563,43cyclictest14574-21taskset20:00:453
2957399553,41cyclictest5704-21taskset21:10:383
2957399553,41cyclictest5704-21taskset21:10:383
2957399553,39cyclictest20740-21taskset20:20:043
2957399553,39cyclictest20740-21taskset20:20:043
2957399553,16cyclictest0-21swapper/322:25:243
2957399552,42cyclictest30258-21taskset20:50:013
2957399552,39cyclictest3162-21taskset22:47:233
2957399552,39cyclictest3162-21taskset22:47:233
2957399543,38cyclictest12554-21chrt19:55:183
2957399534,35cyclictest15910-21taskset21:26:273
2957399534,35cyclictest15910-21taskset21:26:273
2957399533,35cyclictest12767-21chrt23:02:183
2957399533,35cyclictest12767-21chrt23:02:183
2957399532,41cyclictest11290-21chrt21:20:153
2957399532,38cyclictest29723-21chrt00:20:003
2957399532,38cyclictest29723-21chrt00:20:003
2957399524,36cyclictest2776-21taskset19:25:053
2957399524,35cyclictest24920-21chrt00:11:053
2957399523,36cyclictest24398-21chrt20:30:253
2957399523,35cyclictest6708-21chrt00:34:353
2957399523,35cyclictest6708-21chrt00:34:353
2957399513,36cyclictest5330-21taskset22:00:353
2957399513,36cyclictest5330-21taskset22:00:353
2957399513,35cyclictest774-21chrt19:20:033
2957399513,35cyclictest24385-21chrt22:30:213
2957399513,35cyclictest24385-21chrt22:30:213
2957399512,36cyclictest29249-21chrt23:28:273
2957399512,33cyclictest24957-21taskset23:20:413
2957399504,36cyclictest15932-21chrt23:07:123
29573995037,8cyclictest273842sleep322:35:163
2957399503,35cyclictest3141-21chrt00:28:393
2957399503,35cyclictest30241-21chrt00:20:173
2957399503,34cyclictest29043-21chrt21:47:143
2957399503,33cyclictest32141-21chrt20:55:153
2957399503,33cyclictest32141-21chrt20:55:153
2957399503,12cyclictest0-21swapper/323:50:203
2957399502,39cyclictest21947-21taskset00:06:393
2957399502,38cyclictest4213-21taskset22:00:023
2957399502,35cyclictest418-21chrt23:34:303
2957399502,35cyclictest16649-21chrt22:19:153
2957399495,34cyclictest1494-21chrt23:35:283
2957399494,37cyclictest621-21taskset22:44:033
29573994936,8cyclictest451rcuc/321:03:433
2957399493,34cyclictest23339-21chrt23:19:443
2957399493,34cyclictest23339-21chrt23:19:443
2957399493,33cyclictest23794-21chrt20:30:003
2957399493,32cyclictest27026-21chrt20:40:013
2957399492,33cyclictest25313-21chrt21:40:543
2957399483,32cyclictest6815-21chrt22:53:313
2957399482,32cyclictest19825-21chrt21:33:163
2957399473,31cyclictest15296-21chrt23:56:063
29573994728,12cyclictest0-21swapper/322:55:373
2957399464,32cyclictest17804-21chrt00:00:273
2957399463,30cyclictest6047-21chrt23:42:293
2957399462,33cyclictest22232-21chrt20:25:023
2957399462,32cyclictest11794-21taskset22:10:403
2957399453,29cyclictest18261-21chrt20:10:333
2957399452,29cyclictest19123-21chrt22:21:463
2957399433,25cyclictest9272-21chrt19:45:143
2957399432,29cyclictest3765-21chrt21:05:363
2957399432,28cyclictest31077-21chrt19:15:013
2957399423,29cyclictest7595-21taskset23:45:203
2957399423,27cyclictest8474-21chrt22:05:343
2957399423,25cyclictest7815-21taskset19:40:203
29573994230,8cyclictest451rcuc/320:09:453
2957399422,26cyclictest18850-21chrt23:11:293
2957399412,30cyclictest2285-21taskset19:24:533
2957399403,28cyclictest32210-21taskset21:52:083
29573994025,7cyclictest451rcuc/319:50:083
29573994025,7cyclictest451rcuc/319:50:083
2957399393,28cyclictest30669-21taskset20:50:213
2957399393,26cyclictest6704-21taskset19:35:343
2957399393,26cyclictest27630-21taskset20:40:253
2957399383,26cyclictest5288-21taskset19:30:363
2957399353,24cyclictest29612-21taskset19:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional