You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 02:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Tue Dec 16, 2025 01:38:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
983821420,2sleep30-21swapper/319:21:543
536421140,9sleep05367-21sed19:11:460
536421140,9sleep05367-21sed19:11:460
1694721120,2sleep20-21swapper/222:39:302
2793021100,3chrt0-21swapper/320:11:523
2793021100,3chrt0-21swapper/320:11:523
2609821100,4sleep20-21swapper/220:06:532
32522970,3chrt0-21swapper/322:01:503
32522970,3chrt0-21swapper/322:01:503
122572970,8sleep012-21ksoftirqd/022:26:460
204372930,2sleep20-21swapper/200:16:502
211932910,3sleep04943-21cyclictest22:51:450
223982870,3chrt0-21swapper/221:26:452
127092800,6sleep212852-21timerandwakeup22:26:532
22012760,2chrt0-21swapper/020:31:460
224142710,3sleep30-21swapper/300:21:533
224142710,3sleep30-21swapper/300:21:533
4202690,3sleep20-21swapper/223:21:532
4202690,3sleep20-21swapper/223:21:532
218412640,4sleep30-21swapper/322:51:533
48982570,3sleep24668-21ls19:10:292
48982570,3sleep24668-21ls19:10:292
225742530,3sleep30-21swapper/319:56:543
224302510,4sleep222432-21awk19:56:532
108512490,2sleep30-21swapper/322:21:553
108512490,2sleep30-21swapper/322:21:553
147882480,3sleep30-21swapper/319:36:493
48812470,2sleep14668-21ls19:10:161
48812470,2sleep14668-21ls19:10:161
142112410,3sleep00-21swapper/021:01:560
142112410,3sleep00-21swapper/021:01:560
164962390,3sleep30-21swapper/321:10:033
164962390,3sleep30-21swapper/321:10:033
244762380,3sleep10-21swapper/121:31:471
50522370,2sleep20-21swapper/220:36:552
50522370,2sleep20-21swapper/220:36:552
86932360,3sleep20-21swapper/222:16:482
86932360,3sleep20-21swapper/222:16:482
319282360,2sleep232007-21needreboot21:51:492
275852350,2sleep227692-21bingrep20:11:482
275852350,2sleep227692-21bingrep20:11:482
185362350,1sleep218610-21strings00:11:482
185362350,1sleep218610-21strings00:11:482
177552350,3sleep20-21swapper/221:11:542
177552350,3sleep20-21swapper/221:11:542
316592340,3sleep0151rcuc/021:51:460
318352330,3sleep30-21swapper/320:21:573
318352330,3sleep30-21swapper/320:21:573
89702320,3sleep00-21swapper/023:46:460
89702320,3sleep00-21swapper/023:46:460
193332320,2sleep30-21swapper/322:46:463
193332320,2sleep30-21swapper/322:46:463
112502300,2sleep30-21swapper/320:56:453
247972290,3sleep20-21swapper/221:31:512
48992287,15sleep30-21swapper/319:10:303
48992287,15sleep30-21swapper/319:10:303
494499271,17cyclictest24294-21grep21:31:450
259492270,3chrt0-21swapper/020:06:510
4946992614,3cyclictest27-21ksoftirqd/223:56:482
4946992614,3cyclictest27-21ksoftirqd/223:56:482
494599241,20cyclictest24359-21ls20:01:541
494599241,20cyclictest24359-21ls20:01:541
4946992313,3cyclictest27-21ksoftirqd/200:34:162
4946992313,3cyclictest27-21ksoftirqd/200:34:162
4946992312,3cyclictest0-21swapper/223:51:462
494699231,18cyclictest9597-21grep20:51:462
494699231,18cyclictest9597-21grep20:51:462
494599231,17cyclictest13678-21find22:31:451
494599231,17cyclictest13678-21find22:31:451
208502230,3sleep30-21swapper/300:16:553
4946992216,4cyclictest27-21ksoftirqd/222:41:532
4946992216,4cyclictest27-21ksoftirqd/200:01:522
4946992216,4cyclictest261rcuc/220:21:542
4946992216,4cyclictest261rcuc/220:21:542
4946992215,4cyclictest27-21ksoftirqd/200:26:542
494699221,19cyclictest27094-21ls23:06:512
494699221,19cyclictest27094-21ls23:06:512
494699221,19cyclictest11349-21cut20:56:462
494699221,18cyclictest30576-21ls23:16:472
4946992211,3cyclictest27-21ksoftirqd/220:16:522
4945992211,9cyclictest27366-21cron21:40:001
494599221,18cyclictest14734-21ls19:36:481
494499221,17cyclictest28366-21find23:11:440
494499221,11cyclictest30090-21head20:16:560
128552220,2sleep30-21swapper/322:26:533
494699211,11cyclictest31181-21grep21:50:032
494699211,11cyclictest31181-21grep21:50:032
494599211,6cyclictest17453-21ps21:11:521
494599211,6cyclictest17453-21ps21:11:521
494599211,16cyclictest29299-21ls20:16:481
494499211,18cyclictest21286-21ls19:55:010
494499211,18cyclictest1624-21uniq23:26:470
494499211,11cyclictest10841-21fschecks_time23:51:470
494499210,18cyclictest8519-21ls22:16:460
494499210,18cyclictest8519-21ls22:16:460
494499210,18cyclictest24001-21ls00:26:490
494499210,14cyclictest9723-21ls20:51:470
494499210,14cyclictest9723-21ls20:51:470
494799202,14cyclictest23965-21taskset22:56:553
494699207,11cyclictest13651-21tail21:01:502
494699207,11cyclictest13651-21tail21:01:502
494699202,9cyclictest7964-21grep19:16:552
4946992014,4cyclictest27-21ksoftirqd/200:38:092
4946992013,5cyclictest27-21ksoftirqd/220:46:452
4946992013,5cyclictest27-21ksoftirqd/220:46:452
4946992013,4cyclictest27-21ksoftirqd/223:41:452
4946992013,4cyclictest27-21ksoftirqd/223:37:592
4946992013,4cyclictest27-21ksoftirqd/223:37:592
4946992013,4cyclictest27-21ksoftirqd/223:01:302
4946992013,4cyclictest27-21ksoftirqd/223:01:302
4946992013,4cyclictest27-21ksoftirqd/221:41:462
4946992013,4cyclictest27-21ksoftirqd/221:41:462
4946992012,6cyclictest10766-21fschecks_time19:26:462
4946992012,6cyclictest10766-21fschecks_time19:26:462
494699201,16cyclictest26108-21tune2fs21:36:472
494699200,18cyclictest28564-21cut23:11:462
494599201,17cyclictest23760-21cut00:26:481
494599201,17cyclictest18255-21gltestperf19:46:461
494599201,17cyclictest18255-21gltestperf19:46:461
494599200,16cyclictest2651-21find22:01:451
494599200,16cyclictest2651-21find22:01:451
494499206,11cyclictest23945-21latency20:01:480
494499206,11cyclictest23945-21latency20:01:480
494499204,9cyclictest12-21ksoftirqd/023:06:560
494499204,9cyclictest12-21ksoftirqd/023:06:560
494499201,9cyclictest28812-21cat00:40:020
494499201,17cyclictest5749-21needreboot23:36:480
494499201,17cyclictest5749-21needreboot23:36:480
494499201,16cyclictest541-21ls20:26:480
494499201,16cyclictest541-21ls20:26:480
494499201,14cyclictest23430-21missed_timers22:56:490
494499201,13cyclictest1701-21ls21:56:530
494499201,11cyclictest2788-21fschecks_count22:01:470
494499201,11cyclictest2788-21fschecks_count22:01:470
494499201,10cyclictest26121-21fschecks_time21:36:470
494799191,14cyclictest0-21swapper/323:25:033
494799191,14cyclictest0-21swapper/323:25:033
494799191,14cyclictest0-21swapper/323:11:553
494799191,13cyclictest0-21swapper/322:06:463
494699198,9cyclictest24903-21cron20:05:002
494699198,9cyclictest24903-21cron20:05:002
494699196,4cyclictest131rcu_preempt21:21:462
494699196,4cyclictest131rcu_preempt21:21:462
494699192,14cyclictest15964-21grep21:06:552
494699192,14cyclictest15964-21grep21:06:552
4946991912,5cyclictest27-21ksoftirqd/222:49:182
4946991912,5cyclictest27-21ksoftirqd/222:49:182
4946991912,4cyclictest27-21ksoftirqd/222:11:462
4946991912,4cyclictest27-21ksoftirqd/219:36:552
494699191,15cyclictest20025-21ls21:20:002
4946991910,7cyclictest22149-21grep22:51:552
4946991910,7cyclictest10373-21grep22:21:492
4946991910,7cyclictest10373-21grep22:21:492
494599192,5cyclictest21-21ksoftirqd/120:26:561
494599192,5cyclictest21-21ksoftirqd/120:26:561
4945991913,4cyclictest0-21swapper/121:41:441
4945991913,4cyclictest0-21swapper/121:41:441
4945991912,5cyclictest6708-21cat20:41:531
4945991912,5cyclictest0-21swapper/121:51:441
4945991911,7cyclictest19805-21needreboot22:46:501
4945991911,7cyclictest19805-21needreboot22:46:501
4945991911,6cyclictest8393-21cut22:16:451
4945991911,6cyclictest8393-21cut22:16:451
4945991911,6cyclictest32435-21basename23:21:471
4945991911,6cyclictest32435-21basename23:21:471
4945991911,6cyclictest10204-21cron19:25:001
4945991910,6cyclictest18039-21ntpq22:41:511
494499197,4cyclictest0-21swapper/020:36:550
494499197,4cyclictest0-21swapper/020:36:550
494499197,10cyclictest22850-21cut21:26:490
494499196,4cyclictest131rcu_preempt00:11:350
494499196,4cyclictest131rcu_preempt00:11:350
494499194,4cyclictest131rcu_preempt00:31:560
494499194,4cyclictest131rcu_preempt00:31:560
4944991911,6cyclictest6843-21seq19:16:280
494499191,13cyclictest11766-21ls20:56:490
494499190,15cyclictest30781-21ls23:16:490
494499190,14cyclictest22170-21ls00:21:490
494499190,14cyclictest22170-21ls00:21:490
33152191,3sleep20-21swapper/220:32:582
14172191,3sleep20-21swapper/220:27:072
14172191,3sleep20-21swapper/220:27:072
494799181,13cyclictest0-21swapper/300:31:473
494799181,13cyclictest0-21swapper/300:31:473
494799181,13cyclictest0-21swapper/300:11:473
494799181,13cyclictest0-21swapper/300:11:473
494699187,3cyclictest27-21ksoftirqd/222:01:522
494699187,3cyclictest27-21ksoftirqd/222:01:522
494699183,12cyclictest22884-21unixbench_singl00:21:562
494699183,12cyclictest22884-21unixbench_singl00:21:562
494699181,7cyclictest17211-21grep19:41:552
494699181,7cyclictest17211-21grep19:41:552
4946991812,4cyclictest10288-21latency_hist23:50:002
4946991812,4cyclictest10288-21latency_hist23:50:002
4946991811,4cyclictest27-21ksoftirqd/222:31:472
4946991811,4cyclictest27-21ksoftirqd/222:31:472
494699181,14cyclictest20372-21uname19:51:492
494699181,14cyclictest18149-21fschecks_time19:46:462
494699181,14cyclictest18149-21fschecks_time19:46:462
4946991810,5cyclictest27-21ksoftirqd/220:41:512
494599189,7cyclictest22962-21/usr/sbin/munin22:56:451
494599183,12cyclictest5254-21cut23:36:431
494599183,12cyclictest5254-21cut23:36:431
4945991812,4cyclictest27316-21grep20:11:451
4945991812,4cyclictest27316-21grep20:11:451
4945991810,6cyclictest26827-21idleruntime-cro00:35:001
4945991810,6cyclictest26827-21idleruntime-cro00:35:001
494599180,7cyclictest10797-21fschecks_count23:51:471
494499189,7cyclictest31023-21grep20:21:480
494499189,7cyclictest31023-21grep20:21:480
494499189,6cyclictest0-21swapper/022:35:030
494499189,6cyclictest0-21swapper/022:35:030
494499188,8cyclictest13167-21ntpq23:56:510
494499188,8cyclictest13167-21ntpq23:56:510
494499183,13cyclictest18829-21tail21:16:480
494499182,5cyclictest12-21ksoftirqd/000:01:480
494499182,14cyclictest27974-21taskset21:41:460
494499182,14cyclictest27974-21taskset21:41:460
494499181,8cyclictest10295-21grep22:21:480
494499181,8cyclictest10295-21grep22:21:480
494499181,7cyclictest131rcu_preempt23:31:520
4944991812,4cyclictest18544-21grep22:41:560
4944991812,4cyclictest17562-21latency_hist19:45:000
4944991812,4cyclictest17562-21latency_hist19:45:000
4944991811,6cyclictest16160-21/usr/sbin/munin00:06:560
4944991811,6cyclictest0-21swapper/023:01:480
4944991811,6cyclictest0-21swapper/023:01:480
4944991811,5cyclictest4836-21gltestperf22:06:460
494499181,10cyclictest9287-21sort19:21:480
494499180,9cyclictest7520-21/usr/sbin/munin20:46:500
494499180,9cyclictest7520-21/usr/sbin/munin20:46:500
254732180,3sleep30-21swapper/320:06:463
167412181,3sleep10-21swapper/122:36:571
494799171,12cyclictest0-21swapper/320:36:433
494799171,12cyclictest0-21swapper/320:36:433
494699179,6cyclictest16360-21fschecks_time00:06:452
494699175,10cyclictest12554-21fschecks_count19:31:462
494699171,14cyclictest4754-21cut22:06:462
494699171,13cyclictest8975-21cut19:21:452
494699171,13cyclictest3560-21fschecks_time23:31:462
494699171,13cyclictest23091-21grep22:56:462
494699171,13cyclictest1404-21needreboot21:56:482
4946991710,4cyclictest27-21ksoftirqd/223:26:562
494599179,6cyclictest0-21swapper/123:45:001
494599177,8cyclictest3830-21memory23:31:491
494599176,9cyclictest2202-21fschecks_time20:31:461
494599174,4cyclictest131rcu_preempt20:06:461
494599173,12cyclictest18815-21find00:11:521
494599173,12cyclictest18815-21find00:11:521
494599171,3cyclictest131rcu_preempt19:41:551
494599171,3cyclictest131rcu_preempt19:41:551
4945991712,3cyclictest16221-21cpu00:06:431
4945991711,4cyclictest0-21swapper/123:26:511
4945991711,4cyclictest0-21swapper/120:36:451
4945991711,4cyclictest0-21swapper/120:36:451
494599171,13cyclictest22314-21cut00:21:511
494599171,13cyclictest22314-21cut00:21:511
494599171,13cyclictest18898-21sed21:16:481
494599171,12cyclictest8519-21grep19:20:021
494599171,11cyclictest22255-21taskset22:51:571
4945991710,5cyclictest29926-21sed21:46:461
4945991710,5cyclictest29926-21sed21:46:461
4945991710,5cyclictest20081-21fschecks_time00:16:471
494499176,9cyclictest15201-21gltestperf21:06:470
494499176,9cyclictest15201-21gltestperf21:06:470
494499175,4cyclictest12-21ksoftirqd/019:26:490
494499175,4cyclictest12-21ksoftirqd/019:26:490
494499172,12cyclictest28707-21latency_hist20:15:030
494499172,12cyclictest28707-21latency_hist20:15:030
494499172,12cyclictest20703-21grep21:21:460
494499172,12cyclictest20703-21grep21:21:460
494499171,9cyclictest18133-21fschecks_time19:46:460
494499171,9cyclictest18133-21fschecks_time19:46:460
494499171,8cyclictest29984-21if_eth021:46:470
494499171,8cyclictest29984-21if_eth021:46:470
494499171,7cyclictest6499-21fschecks_count22:11:470
4944991711,4cyclictest0-21swapper/000:17:300
494499171,13cyclictest19357-21fschecks_count22:46:460
494499171,13cyclictest19357-21fschecks_count22:46:460
494499170,14cyclictest12453-21cpuspeed19:31:450
494799168,6cyclictest33-21ksoftirqd/321:34:273
494799161,13cyclictest0-21swapper/320:46:493
494799161,13cyclictest0-21swapper/320:46:493
494799161,12cyclictest0-21swapper/320:26:563
494799161,12cyclictest0-21swapper/320:26:563
494799161,12cyclictest0-21swapper/300:26:503
494599169,5cyclictest13094-21seq22:27:101
494599169,5cyclictest0-21swapper/120:46:451
494599169,5cyclictest0-21swapper/120:46:451
494599168,6cyclictest9744-21switchtime23:46:541
494599168,6cyclictest9744-21switchtime23:46:541
494599168,6cyclictest10370-21gltestperf22:21:491
494599168,6cyclictest10370-21gltestperf22:21:491
494599163,11cyclictest780-21/usr/sbin/munin21:56:491
494599163,10cyclictest28823-21uname23:11:481
494599162,12cyclictest4752-21tune2fs22:06:461
4945991611,3cyclictest22657-21interrupts21:26:471
494599161,13cyclictest20681-21ls19:51:531
494599161,13cyclictest14973-21df_inode21:06:451
494599161,13cyclictest14973-21df_inode21:06:451
494599161,13cyclictest10882-21gltestperf19:26:471
494599161,13cyclictest10882-21gltestperf19:26:471
494599161,12cyclictest31014-21fschecks_time20:21:481
494599161,12cyclictest31014-21fschecks_time20:21:481
494599161,12cyclictest30501-21grep23:16:471
494599161,12cyclictest22024-21kernelversion19:56:471
494599161,12cyclictest20556-21grep21:21:461
494599161,12cyclictest20556-21grep21:21:461
494599161,12cyclictest14680-21grep00:01:471
494599161,12cyclictest13323-21grep21:01:471
494599161,12cyclictest13323-21grep21:01:471
4945991610,4cyclictest0-21swapper/120:56:561
494599160,14cyclictest12695-21gltestperf19:31:471
494499163,10cyclictest16881-21fschecks_count21:11:460
494499163,10cyclictest16881-21fschecks_count21:11:460
494499161,8cyclictest697-21runrttasks19:35:310
494499161,13cyclictest0-21swapper/023:20:490
494499161,13cyclictest0-21swapper/023:20:490
494499161,12cyclictest7558-21wc23:41:490
494499161,12cyclictest5931-21grep20:41:450
494499161,12cyclictest467-21cron22:40:000
494799159,4cyclictest33-21ksoftirqd/323:46:173
494799159,4cyclictest33-21ksoftirqd/323:46:173
494799159,4cyclictest33-21ksoftirqd/300:00:073
494799156,6cyclictest33-21ksoftirqd/321:41:063
494799156,6cyclictest33-21ksoftirqd/321:41:063
494799152,11cyclictest23320-21chrt21:26:543
494799151,9cyclictest0-21swapper/321:01:553
494799151,9cyclictest0-21swapper/321:01:553
494799151,12cyclictest0-21swapper/323:41:483
494799151,12cyclictest0-21swapper/321:51:553
494799151,12cyclictest0-21swapper/320:51:463
494799151,12cyclictest0-21swapper/320:51:463
494799151,12cyclictest0-21swapper/320:16:463
494799151,11cyclictest20899-21chrt21:21:483
494799151,11cyclictest20899-21chrt21:21:483
494599159,4cyclictest25270-21missed_timers23:01:491
494599159,4cyclictest25270-21missed_timers23:01:491
494599152,11cyclictest697-21runrttasks00:35:261
494599151,12cyclictest6892-21missed_timers22:11:491
494599151,11cyclictest9580-21grep20:51:461
494599151,11cyclictest9580-21grep20:51:461
494599151,11cyclictest697-21runrttasks23:06:461
494599151,11cyclictest697-21runrttasks23:06:461
494599151,11cyclictest12602-21fschecks_count23:56:461
494599151,11cyclictest12602-21fschecks_count23:56:461
494499151,12cyclictest21730-21grep19:56:450
35872150,3sleep30-21swapper/323:31:463
494799149,4cyclictest33-21ksoftirqd/323:39:503
494799149,4cyclictest33-21ksoftirqd/323:39:503
494799149,4cyclictest33-21ksoftirqd/322:12:393
494799149,4cyclictest33-21ksoftirqd/321:58:123
494799148,4cyclictest70262chrt20:43:153
494799148,4cyclictest33-21ksoftirqd/321:15:533
494799148,4cyclictest33-21ksoftirqd/319:30:223
494799147,5cyclictest33-21ksoftirqd/322:38:313
494799147,5cyclictest33-21ksoftirqd/319:43:253
494799147,5cyclictest33-21ksoftirqd/319:43:253
494799146,6cyclictest33-21ksoftirqd/323:27:463
494799144,7cyclictest33-21ksoftirqd/300:36:483
494799141,8cyclictest0-21swapper/323:06:563
494799141,8cyclictest0-21swapper/323:06:563
494799141,10cyclictest14044-21chrt22:31:473
494799141,10cyclictest14044-21chrt22:31:473
494799138,4cyclictest33-21ksoftirqd/323:55:173
494799138,4cyclictest33-21ksoftirqd/323:55:173
494799138,4cyclictest33-21ksoftirqd/323:53:443
494799138,4cyclictest33-21ksoftirqd/320:01:403
494799138,4cyclictest33-21ksoftirqd/320:01:403
494799138,4cyclictest33-21ksoftirqd/319:48:463
494799138,4cyclictest33-21ksoftirqd/319:48:463
494799138,3cyclictest118402chrt19:27:323
494799138,3cyclictest118402chrt19:27:323
494799137,4cyclictest171372sleep321:11:483
494799137,4cyclictest171372sleep321:11:483
494799134,6cyclictest0-21swapper/321:40:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional