You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-02 - 09:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7.osadl.org (updated Sun Nov 02, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
82934299613,35cyclictest0-21swapper/000:08:090
829343995819,14cyclictest11050irq/107-eth%d00:05:471
829342995022,12cyclictest1031129-21sendmail-msp00:11:220
82934499475,13cyclictest1032130-21ssh00:07:062
829342994514,14cyclictest948289-21ssh22:37:570
829342994313,13cyclictest14-21ksoftirqd/021:57:570
82934399407,25cyclictest0-21swapper/122:27:491
82934399404,16cyclictest947290-21/usr/sbin/munin22:37:571
82934399355,25cyclictest619-21mta-sts-daemon21:51:431
82934299342,7cyclictest11050irq/107-eth%d23:33:000
82934299342,27cyclictest956979-21telnet22:47:490
82934499335,5cyclictest171rcu_preempt21:42:532
82934299334,13cyclictest0-21swapper/020:42:560
82934399322,5cyclictest829519-21grep19:15:401
82934299322,27cyclictest0-21swapper/000:30:110
82934299321,4cyclictest952653-21irqstats22:42:520
112699320,5rtkit-daemon1125-21rtkit-daemon22:50:232
82934499312,7cyclictest171rcu_preempt00:02:542
82934399311,13cyclictest837793-21sed19:38:011
82934299312,29cyclictest0-21swapper/022:07:590
829343993015,13cyclictest886228-21cut21:32:501
829343993015,13cyclictest886228-21cut21:32:501
829342993019,9cyclictest884135-21ssh21:30:170
829342993019,9cyclictest884135-21ssh21:30:170
82934399291,9cyclictest840089-21fschecks_count19:47:511
82934399291,6cyclictest830009-21gltestperf19:17:531
82934299297,3cyclictest980519-21ssh23:12:500
82934299291,18cyclictest1014511-21ssh23:47:530
82934499286,3cyclictest171rcu_preempt23:27:552
82934399283,8cyclictest884135-21ssh21:30:171
82934399283,8cyclictest884135-21ssh21:30:171
82934399282,8cyclictest863856-21grep20:57:491
82934399282,10cyclictest910409-21collect221:57:571
82934399281,7cyclictest848743-21grep20:12:581
82934399281,5cyclictest957374-21ssh22:47:521
82934399280,9cyclictest863330-21idleruntime-cro20:55:401
82934399280,11cyclictest834583-21aten2.4-expect19:32:481
82934299283,2cyclictest846836-21grep20:07:530
82934399278,2cyclictest0-21swapper/100:17:521
82934399275,8cyclictest993315-21sendmail-msp23:25:481
82934399273,8cyclictest870533-21ssh21:15:431
82934399273,5cyclictest936310-21cron22:25:401
82934399272,8cyclictest111788-21systemd-journal21:40:401
82934399271,5cyclictest966469-21fschecks_count22:57:511
82934399271,5cyclictest11050irq/107-eth%d21:42:531
829342992716,4cyclictest898665-21latency_hist21:45:410
829342992715,1cyclictest0-21swapper/021:31:250
829342992715,1cyclictest0-21swapper/021:31:250
82934499266,15cyclictest1039865-21unin-run00:15:402
82934399264,4cyclictest1000115-21grep23:32:551
82934399263,6cyclictest990248-21/usr/sbin/munin23:22:491
82934399262,6cyclictest0-21swapper/119:52:491
829343992617,8cyclictest851037-21unixbench_singl20:18:011
82934399261,6cyclictest845778-21/usr/sbin/munin20:03:001
829343992611,8cyclictest1008-21mta-sts-daemon00:11:541
82934299262,4cyclictest11050irq/107-eth%d23:25:480
829342992622,2cyclictest966653-21/usr/sbin/munin22:57:520
82934299262,19cyclictest0-21swapper/021:12:550
82934299262,18cyclictest0-21swapper/023:03:010
82934299260,26cyclictest69-21kswapd021:15:430
82934299260,26cyclictest0-21swapper/021:22:480
82934299260,20cyclictest0-21swapper/023:17:470
82934399252,5cyclictest924577-21ssh22:12:551
82934399252,5cyclictest853376-21aten2.4-expect20:27:491
82934399252,5cyclictest1009778-21/usr/sbin/munin23:42:521
82934399252,3cyclictest901609-21grep21:48:021
82934399251,9cyclictest1028542-21cat00:02:521
82934399251,8cyclictest1057065-21fschecks_count00:37:541
82934399251,7cyclictest764-21rs:main1
82934399251,6cyclictest838583-21tr19:42:531
82934399251,6cyclictest832530-21grep19:22:591
82934299251,19cyclictest1028743-21ssh00:02:540
82934499247,11cyclictest1010867-21ssh23:43:272
829344992417,5cyclictest1056831-21aten_r2power_fr00:37:522
829344992417,5cyclictest1006-21mta-sts-daemon23:33:002
82934399249,13cyclictest271ktimers/120:40:541
82934399244,5cyclictest0-21swapper/123:17:511
82934399243,6cyclictest846534-21aten_r2power_po20:07:501
82934399242,7cyclictest1048360-21sendmail-msp00:28:021
82934399242,5cyclictest851657-21fschecks_time20:22:531
82934399242,3cyclictest11050irq/107-eth%d23:52:511
82934399241,4cyclictest866634-21sendmail-msp21:07:581
829343992413,7cyclictest762-21in:imuxsock23:13:011
82934299242,6cyclictest14-21ksoftirqd/022:02:580
82934299242,3cyclictest942591-21/usr/sbin/munin22:32:570
82934299240,5cyclictest990124-21aten_r2power_cu23:22:480
82934499234,14cyclictest870526-21apt-get21:15:432
82934499233,14cyclictest980526-21df_inode23:12:502
82934499232,14cyclictest1019490-21ntpq23:52:572
829344992320,2cyclictest972267-21unixbench_multi23:03:022
82934399235,4cyclictest1015188-21/usr/sbin/munin23:48:011
82934399233,5cyclictest0-21swapper/122:53:041
82934399233,3cyclictest975567-21/usr/sbin/munin23:07:531
82934399233,15cyclictest1023172-21/usr/sbin/munin23:58:001
82934399232,7cyclictest972025-21sh23:03:011
82934399232,6cyclictest111788-21systemd-journal20:00:401
82934399232,5cyclictest856617-21/usr/sbin/munin20:37:561
82934399231,6cyclictest1060377-21aten_r2power_cu00:42:491
82934399231,5cyclictest919397-21grep22:07:541
82934299231,20cyclictest1021655-21latency_hist23:55:400
82934399223,5cyclictest1046462-21/usr/sbin/munin00:22:561
82934399223,4cyclictest1004733-21grep23:37:511
82934399223,3cyclictest914856-21/usr/sbin/munin22:02:521
82934399222,8cyclictest111788-21systemd-journal21:12:511
82934399222,6cyclictest952199-21/usr/sbin/munin22:42:491
82934399221,7cyclictest111788-21systemd-journal00:32:561
82934299225,3cyclictest14-21ksoftirqd/020:02:550
82934299222,19cyclictest191rcuc/020:57:530
82934299221,20cyclictest1058069-21ssh00:38:040
82934299221,10cyclictest0-21swapper/022:52:580
82934299220,10cyclictest0-21swapper/021:52:560
82934499217,10cyclictest1008-21mta-sts-daemon21:12:522
82934499214,15cyclictest901118-21ntpq21:47:572
82934499213,14cyclictest886613-21rm21:32:532
82934499213,14cyclictest886613-21rm21:32:532
82934399213,3cyclictest865309-21/usr/sbin/munin21:02:491
82934399213,12cyclictest11050irq/107-eth%d22:17:561
82934399211,6cyclictest834392-21grep19:30:461
82934299213,5cyclictest14-21ksoftirqd/019:38:000
82934299212,4cyclictest1041212-21/usr/sbin/munin00:18:020
829342992115,6cyclictest111788-21systemd-journal20:52:580
82934299211,4cyclictest0-21swapper/019:22:560
82934499205,12cyclictest1005-21mta-sts-daemon20:25:432
829344992016,2cyclictest1006-21mta-sts-daemon00:42:552
82934499201,3cyclictest0-21swapper/220:42:562
82934399205,3cyclictest941877-21ssh22:31:351
82934399201,5cyclictest111788-21systemd-journal20:47:591
82934299204,15cyclictest151ktimers/020:12:560
829342992017,2cyclictest0-21swapper/023:37:520
829342992013,5cyclictest0-21swapper/021:07:530
82934299201,1cyclictest58-21kcompactd022:17:560
82934599193,8cyclictest0-21swapper/321:27:493
82934599193,8cyclictest0-21swapper/321:27:493
82934599192,8cyclictest0-21swapper/300:05:443
82934499194,11cyclictest1048841-21ssh00:26:252
82934499193,13cyclictest111788-21systemd-journal20:52:592
82934499193,12cyclictest881309-21tail21:27:492
82934499193,12cyclictest881309-21tail21:27:492
82934299195,11cyclictest1007-21mta-sts-daemon00:33:580
829342991913,4cyclictest850101-21gltestperf20:17:510
82934299191,17cyclictest892176-21/usr/sbin/munin21:38:040
82934299191,17cyclictest841447-21/usr/sbin/munin19:52:470
82934299190,13cyclictest975860-21df_inode23:07:510
82934599183,6cyclictest0-21swapper/300:25:533
82934599182,7cyclictest0-21swapper/300:37:543
82934499184,12cyclictest919262-21/usr/sbin/munin22:07:512
82934499182,1cyclictest0-21swapper/223:48:282
829344991815,2cyclictest988213-21systemd-run23:20:412
82934399182,4cyclictest854978-21/usr/sbin/munin20:33:011
82934299186,7cyclictest937967-21ssh22:27:490
82934299185,11cyclictest839556-21latency_hist19:45:400
829342991818,0cyclictest0-21swapper/021:47:530
82934299181,10cyclictest1024093-21ntpq23:57:550
82934599174,7cyclictest0-21swapper/300:02:543
82934599173,7cyclictest0-21swapper/300:42:573
82934499175,8cyclictest892173-21ssh21:38:042
82934499174,9cyclictest916576-21sh22:04:242
82934499174,10cyclictest963538-21ssh22:53:562
82934499173,10cyclictest857186-21/usr/sbin/munin20:37:532
829344991711,5cyclictest321irq_work/220:02:572
82934499170,11cyclictest974035-21sendmail-msp23:07:502
82934299176,2cyclictest0-21swapper/022:12:450
829342991716,1cyclictest0-21swapper/022:22:550
829342991712,3cyclictest840703-21/usr/sbin/munin19:47:580
82934299171,11cyclictest829301-21munin-run19:15:400
82934599163,7cyclictest0-21swapper/323:47:553
82934599163,7cyclictest0-21swapper/323:05:403
82934599163,7cyclictest0-21swapper/321:12:563
82934599163,5cyclictest0-21swapper/322:33:003
82934499164,8cyclictest1024544-21/usr/sbin/munin23:58:002
82934499163,11cyclictest852463-21timerandwakeup20:23:032
82934499162,4cyclictest341rcuc/221:57:572
82934299166,1cyclictest0-21swapper/019:27:580
82934299165,3cyclictest843389-21fschecks_count19:57:510
829342991612,2cyclictest111788-21systemd-journal20:27:550
82934599153,8cyclictest0-21swapper/323:25:483
82934599153,6cyclictest0-21swapper/321:52:573
82934599152,7cyclictest0-21swapper/323:52:563
82934499159,4cyclictest0-21swapper/221:52:542
82934499153,8cyclictest969350-21latency_hist23:00:412
82934499153,11cyclictest1042648-21/usr/sbin/munin00:18:022
82934499152,9cyclictest844412-21/usr/sbin/munin19:58:012
82934499151,9cyclictest948801-21ssh22:38:022
829344991512,2cyclictest0-21swapper/220:47:532
82934499151,10cyclictest0-21swapper/223:20:442
82934299154,2cyclictest0-21swapper/020:32:500
82934299153,9cyclictest852275-21/usr/sbin/munin20:23:000
82934299152,9cyclictest860056-21sendmail-msp20:47:570
82934299152,2cyclictest171rcu_preempt00:43:190
82934299152,12cyclictest14-21ksoftirqd/023:42:500
82934599145,5cyclictest0-21swapper/321:15:433
82934599144,5cyclictest0-21swapper/321:32:533
82934599144,5cyclictest0-21swapper/321:32:533
82934599144,5cyclictest0-21swapper/319:47:553
82934599143,6cyclictest0-21swapper/323:17:553
82934599142,6cyclictest0-21swapper/323:20:443
82934599142,5cyclictest0-21swapper/323:42:093
82934599142,5cyclictest0-21swapper/320:47:583
82934499149,2cyclictest0-21swapper/222:42:542
82934499145,7cyclictest923728-21ssh22:12:472
82934499144,8cyclictest1003736-21ssh23:36:462
82934499144,5cyclictest1053482-21fschecks_time00:32:552
82934499143,8cyclictest933147-21aten_r2power_cu22:22:482
82934499142,9cyclictest1046088-21ssh00:22:522
82934499142,10cyclictest938542-21/usr/sbin/munin22:27:572
82934499142,10cyclictest837852-21sed19:38:012
82934499142,10cyclictest832622-21sendmail19:25:412
82934499141,10cyclictest841959-21if_end019:52:522
82934599134,4cyclictest0-21swapper/322:57:103
82934599133,6cyclictest0-21swapper/322:18:003
82934599133,5cyclictest0-21swapper/300:32:203
82934599133,5cyclictest0-21swapper/300:11:543
82934599132,6cyclictest0-21swapper/323:57:483
82934599132,6cyclictest0-21swapper/321:37:173
82934599131,7cyclictest0-21swapper/323:37:513
82934599131,7cyclictest0-21swapper/321:47:573
82934599131,6cyclictest0-21swapper/320:52:553
82934499139,3cyclictest868125-21unixbench-2d21:08:032
82934499139,3cyclictest836199-21latency_hist19:35:412
82934499135,5cyclictest840156-21tune2fs19:47:522
82934499134,6cyclictest928349-21/usr/sbin/munin22:17:482
82934499133,7cyclictest838729-21kernelversion19:42:542
82934499132,8cyclictest850625-21wc20:17:572
82934499132,8cyclictest829557-21/usr/sbin/munin19:17:542
82934499131,9cyclictest876930-21ssh21:22:512
82934499130,10cyclictest848452-21head20:12:562
82934299137,5cyclictest191rcuc/019:17:510
82934299131,8cyclictest0-21swapper/000:22:560
829342991310,2cyclictest191rcuc/021:02:540
82934299130,12cyclictest834871-21tune2fs19:32:510
82934599125,4cyclictest0-21swapper/321:02:543
82934599123,5cyclictest0-21swapper/320:25:503
82934599123,4cyclictest0-21swapper/323:31:093
82934599123,4cyclictest0-21swapper/322:47:493
82934599123,4cyclictest0-21swapper/320:15:403
82934599122,5cyclictest0-21swapper/322:27:493
82934599122,5cyclictest0-21swapper/320:17:523
82934599122,4cyclictest0-21swapper/321:56:013
82934499128,3cyclictest118475-21wpa_supplicant20:57:532
82934499128,2cyclictest937598-21kworker/u8:2+xprtiod22:31:032
82934499124,5cyclictest847525-21/usr/sbin/munin20:08:022
82934499122,8cyclictest865504-21grep21:02:512
82934499122,7cyclictest834277-21munin-plugin-st19:30:412
82934299122,9cyclictest858296-21cp20:40:410
82934599115,3cyclictest0-21swapper/319:17:573
82934599114,5cyclictest0-21swapper/322:12:503
82934599114,4cyclictest0-21swapper/322:52:493
82934599114,3cyclictest0-21swapper/323:07:513
82934599113,4cyclictest0-21swapper/322:37:543
82934599113,4cyclictest0-21swapper/321:22:113
82934599113,4cyclictest0-21swapper/320:07:583
82934599112,6cyclictest0-21swapper/321:42:533
82934599112,5cyclictest0-21swapper/300:17:173
82934599112,4cyclictest0-21swapper/321:07:503
82934599111,7cyclictest0-21swapper/323:12:583
82934599111,7cyclictest0-21swapper/322:22:503
82934599111,5cyclictest0-21swapper/322:42:503
82934599111,4cyclictest0-21swapper/322:06:463
82934599110,6cyclictest0-21swapper/320:37:533
82934599110,5cyclictest0-21swapper/320:42:483
82934499117,3cyclictest855323-21fschecks_time20:32:532
82934499112,2cyclictest171rcu_preempt19:15:402
82934599104,4cyclictest0-21swapper/319:42:533
82934599104,2cyclictest0-21swapper/320:32:593
82934599104,2cyclictest0-21swapper/320:02:483
82934599103,4cyclictest0-21swapper/322:02:513
82934599103,4cyclictest0-21swapper/320:22:583
82934599103,3cyclictest0-21swapper/319:22:483
82934599102,4cyclictest0-21swapper/300:22:493
8293459994,4cyclictest0-21swapper/319:57:503
8293459992,4cyclictest0-21swapper/319:27:493
8293459992,3cyclictest0-21swapper/320:57:553
8293459991,5cyclictest0-21swapper/319:35:433
8293459982,4cyclictest0-21swapper/319:34:143
8293459981,5cyclictest0-21swapper/319:52:503
8293459980,4cyclictest0-21swapper/319:15:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional