You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 22:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7.osadl.org (updated Fri Nov 28, 2025 12:43:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
269293899511,34cyclictest0-21swapper/010:13:100
269293899504,22cyclictest2692936-21cyclictest07:16:030
269293899481,26cyclictest2825285-21latency_hist11:11:040
269293899413,35cyclictest0-21swapper/009:21:060
269293899404,25cyclictest0-21swapper/012:38:080
2692938994029,9cyclictest764-21rs:main0
269293899392,36cyclictest2868234-21sendmail-msp12:08:130
269293999362,14cyclictest1006-21mta-sts-daemon07:16:041
269293999353,11cyclictest764-21rs:main1
269293899345,16cyclictest0-21swapper/010:18:130
2692938993424,8cyclictest2693486-21systemd07:16:040
2692940993310,6cyclictest11050irq/107-eth%d09:43:092
269293999339,6cyclictest11050irq/107-eth%d10:38:041
269293999333,10cyclictest1-21systemd07:16:031
2692939993316,15cyclictest0-21swapper/109:21:061
2692938993221,10cyclictest2757018-21sendmail-msp09:47:010
269293999314,8cyclictest2769877-21aten_r2power_fr10:03:011
269293999312,10cyclictest2829247-21ssh11:16:061
2692938993022,5cyclictest0-21swapper/010:51:220
269293899292,24cyclictest11650irq/162-mmc011:13:000
2692939992815,1cyclictest0-21swapper/111:48:071
269293899284,3cyclictest0-21swapper/010:28:110
269293899282,21cyclictest2706860-21cpu07:57:590
269293899281,5cyclictest2750234-21/usr/sbin/munin09:38:130
269294099273,16cyclictest11050irq/107-eth%d10:16:072
2692938992724,2cyclictest0-21swapper/011:31:250
2692938992719,5cyclictest0-21swapper/009:18:020
269293899271,25cyclictest0-21swapper/011:53:400
269293899271,11cyclictest274792050ssh09:33:170
269293999267,13cyclictest2781272-21sh10:16:071
269293999266,10cyclictest2743891-21ssh09:28:381
269293899262,23cyclictest2881656-21aten2.4-expect12:23:000
269293899262,23cyclictest0-21swapper/009:53:150
112699260,3rtkit-daemon1125-21rtkit-daemon08:06:200
2692939992514,7cyclictest2868207-21munin-run12:06:041
269293899257,17cyclictest2814924-21gltestperf10:58:020
2692940992419,2cyclictest0-21swapper/207:16:042
269293999242,12cyclictest2805302-21sendmail-msp10:51:261
2692939992421,2cyclictest2730086-21grep09:08:051
269293899242,17cyclictest2900989-21cat12:46:040
269294099237,10cyclictest2869919-21cat12:08:082
2692940992315,5cyclictest762-21in:imuxsock08:33:202
269293999232,14cyclictest619-21mta-sts-daemon09:12:131
269293899234,16cyclictest122792-21ntpd09:13:070
269293899233,19cyclictest0-21swapper/010:48:010
269293899232,15cyclictest633-21gpm10:38:110
269293899231,2cyclictest0-21swapper/012:21:040
269293899231,21cyclictest0-21swapper/007:28:040
112699230,8rtkit-daemon1008-21mta-sts-daemon12:29:142
269293999222,17cyclictest1006-21mta-sts-daemon11:28:061
269293899223,17cyclictest0-21swapper/009:27:590
2692938992212,1cyclictest0-21swapper/008:18:060
2692940992114,5cyclictest2715292-21fschecks_time08:23:042
2692940992110,8cyclictest2769883-21cut10:03:012
2692940992110,7cyclictest2757018-21sendmail-msp09:49:102
269293999218,9cyclictest2872928-21ssh12:11:591
269293999217,9cyclictest2759841-21ssh09:48:381
269293999214,9cyclictest2746466-21/usr/sbin/munin09:33:001
269293999214,12cyclictest2734397-21ssh09:18:051
2692939992111,7cyclictest2789033-21sendmail-msp10:31:071
2692938992115,2cyclictest0-21swapper/011:36:060
269293899210,1cyclictest0-21swapper/012:13:060
269294099207,6cyclictest2893999-21ssh12:38:082
269294099203,3cyclictest111788-21systemd-journal10:50:182
2692940992016,3cyclictest2710274-21grep08:08:022
2692940992016,2cyclictest0-21swapper/211:03:072
2692940992016,2cyclictest0-21swapper/210:31:072
2692939992014,5cyclictest2826233-21ssh11:13:001
269293999201,10cyclictest2894885-21ssh12:38:411
2692939992010,6cyclictest0-21swapper/110:08:131
269293899202,3cyclictest2786069-21/usr/sbin/munin10:23:040
2692938992015,3cyclictest2719559-21grep08:33:210
269293899200,19cyclictest0-21swapper/011:58:060
112699200,4rtkit-daemon1125-21rtkit-daemon12:31:201
269294099193,9cyclictest2862873-21ssh11:58:312
269294099192,10cyclictest11050irq/107-eth%d09:38:102
2692940991915,3cyclictest2724499-21grep08:48:142
2692940991915,2cyclictest2713038-21idleruntime-cro08:16:032
269293999196,9cyclictest1007-21mta-sts-daemon09:41:061
269293899192,8cyclictest2702683-21grep07:43:100
2692938991912,5cyclictest2884407-21sendmail-msp12:33:080
2692938991910,1cyclictest0-21swapper/010:31:060
269294199183,9cyclictest0-21swapper/312:28:083
269294099184,10cyclictest2825282-21latency_hist11:11:042
269294099183,11cyclictest2898659-21ntp_states12:43:082
2692940991813,4cyclictest2854243-21ntp_kernel_pll_11:48:112
269293999185,9cyclictest1005-21mta-sts-daemon09:58:101
269293999182,13cyclictest2858099-21ntpq11:53:081
2692939991813,4cyclictest2704796-21cron07:51:041
2692939991811,6cyclictest1008-21mta-sts-daemon11:33:071
269293999181,12cyclictest2717495-21/usr/sbin/munin08:28:081
269293899184,9cyclictest2709762-21sendmail-msp08:13:140
269293899182,8cyclictest2850005-21fschecks_count11:43:040
2692938991814,1cyclictest2856101-21ssh11:51:040
269293899180,7cyclictest0-21swapper/009:41:060
269293899180,12cyclictest0-21swapper/009:58:090
112699180,0rtkit-daemon0-21swapper/110:48:261
269294199174,10cyclictest0-21swapper/311:11:043
269294199173,9cyclictest0-21swapper/308:28:083
269294199172,8cyclictest0-21swapper/312:38:083
269294099176,7cyclictest2815312-21ntpq10:58:052
269294099175,9cyclictest1007-21mta-sts-daemon09:58:102
269294099175,7cyclictest2716579-21/usr/sbin/munin08:28:082
269294099172,8cyclictest2881670-21/usr/sbin/munin12:23:012
269294099172,14cyclictest2734527-21grep09:18:062
269293999176,6cyclictest111788-21systemd-journal12:08:131
269293999174,7cyclictest11050irq/107-eth%d12:43:061
269293999173,9cyclictest2716361-21sendmail-msp08:33:201
2692939991712,4cyclictest2243333-21munin-node10:22:451
2692938991716,1cyclictest0-21swapper/011:03:080
269293899170,10cyclictest0-21swapper/010:46:010
269294199163,7cyclictest0-21swapper/308:33:073
269294099167,6cyclictest0-21swapper/209:31:082
269294099165,8cyclictest2874361-21/usr/sbin/munin12:13:092
269294099165,8cyclictest2696594-21sendmail-msp07:28:062
269294099165,6cyclictest2727433-21/usr/sbin/munin08:58:082
269294099164,8cyclictest0-21swapper/211:13:132
269294099164,2cyclictest11050irq/107-eth%d11:18:012
269294099162,9cyclictest607-21dbus-daemon07:42:262
2692940991612,3cyclictest2798883-21fschecks_time10:38:002
2692940991612,3cyclictest0-21swapper/207:48:122
2692940991610,5cyclictest2778910-21sendmail_mailst10:13:132
269293999168,6cyclictest1125-21rtkit-daemon11:08:051
269293999163,10cyclictest2819253-21/usr/sbin/munin11:03:081
269293999162,12cyclictest1005-21mta-sts-daemon09:38:081
269293999162,11cyclictest2846862-21ssh11:38:141
2692939991613,1cyclictest0-21swapper/107:38:061
2692939991612,3cyclictest0-21swapper/109:03:101
2692939991610,5cyclictest2711351-21grep08:08:151
269293899165,10cyclictest2730125-21fschecks_time09:08:050
269293899163,8cyclictest2884756-21ssh12:26:210
269293899163,11cyclictest2708328-21/usr/sbin/munin08:02:590
269293899161,2cyclictest2868199-21apt12:06:040
2692938991612,3cyclictest2832264-21latency_hist11:21:040
2692938991612,2cyclictest2721186-21grep08:38:090
269293899161,14cyclictest0-21swapper/010:02:420
2692938991610,5cyclictest151ktimers/007:38:020
2692938991610,5cyclictest0-21swapper/008:53:000
269293899160,15cyclictest2698821-21cut07:33:030
112699160,0rtkit-daemon0-21swapper/212:03:282
269294199154,5cyclictest0-21swapper/310:18:023
269294199153,7cyclictest0-21swapper/310:08:103
269294199153,7cyclictest0-21swapper/309:21:063
269294099155,6cyclictest2844171-21apt-get11:36:062
269294099155,5cyclictest0-21swapper/212:33:042
269294099154,9cyclictest2743060-21ntpq09:28:092
269294099154,7cyclictest0-21swapper/210:08:102
269294099151,9cyclictest967-21snmpd10:51:542
2692940991512,2cyclictest2739668-21unixbench_singl09:23:122
2692940991512,2cyclictest0-21swapper/208:46:032
2692940991512,2cyclictest0-21swapper/208:38:032
269294099151,11cyclictest2793351-21sed10:31:042
2692940991510,3cyclictest2858639-21grep11:53:172
269293999155,4cyclictest11050irq/107-eth%d12:23:031
269293999154,5cyclictest2713969-21/usr/sbin/munin08:18:051
269293999152,9cyclictest2726320-21cat08:56:041
269293999152,11cyclictest1005-21mta-sts-daemon09:53:101
269293999151,5cyclictest11050irq/107-eth%d08:48:021
269293999151,13cyclictest1008-21mta-sts-daemon11:21:071
269293999151,11cyclictest2877761-21aten_r2power_po12:18:001
269293899156,7cyclictest2722444-21/usr/sbin/munin08:43:080
269293899155,8cyclictest2773891-21head10:08:010
2692938991512,2cyclictest2703632-21cpuspeed07:48:020
2692938991510,3cyclictest2729384-21grep09:03:350
269293899150,10cyclictest2726804-21fschecks_time08:58:020
112699150,3rtkit-daemon1006-21mta-sts-daemon08:26:410
112699150,2rtkit-daemon1125-21rtkit-daemon11:23:350
269294199144,6cyclictest0-21swapper/310:13:113
269294199144,3cyclictest0-21swapper/312:23:013
269294199143,5cyclictest0-21swapper/311:13:083
269294199142,7cyclictest0-21swapper/307:48:103
269294099148,4cyclictest0-21swapper/211:23:022
269294099144,8cyclictest0-21swapper/207:38:002
269294099142,7cyclictest2761942-21ssh09:53:012
269294099142,10cyclictest2699650-21/usr/sbin/munin07:33:152
269294099141,9cyclictest2786789-21/usr/sbin/munin10:23:052
2692940991411,2cyclictest2878118-21gltestperf12:18:022
269294099141,12cyclictest2714343-21grep08:18:122
269294099141,11cyclictest2243333-21munin-node11:43:012
269293999148,5cyclictest2707463-21ntp_kernel_err07:58:031
269293999148,4cyclictest0-21swapper/110:11:231
269293999145,6cyclictest2715894-21/usr/sbin/munin08:23:101
269293999143,8cyclictest2850685-21ssh11:43:111
269293999142,6cyclictest2790466-21/usr/sbin/munin10:28:041
269293999142,5cyclictest2802885-21fschecks_time10:43:021
269293999141,10cyclictest2816182-21sh10:58:371
269293999140,10cyclictest2720312-21sed08:38:001
269293899142,7cyclictest2724490-21/usr/sbin/munin08:48:140
269293899142,7cyclictest2706453-21munin-plugin-st07:56:030
269293899140,9cyclictest2716184-21sed08:23:140
269294199133,6cyclictest0-21swapper/309:29:103
269294199133,4cyclictest0-21swapper/311:23:073
269294199133,4cyclictest0-21swapper/307:23:023
269294199132,8cyclictest0-21swapper/309:13:073
269294199132,6cyclictest0-21swapper/311:36:063
269294199132,6cyclictest0-21swapper/309:43:123
269294199132,6cyclictest0-21swapper/308:43:023
269294199132,4cyclictest0-21swapper/308:58:023
269294199131,9cyclictest0-21swapper/311:16:063
269294199131,7cyclictest0-21swapper/312:13:163
269294199131,5cyclictest0-21swapper/311:53:043
269294099135,6cyclictest2729522-21sendmail-msp09:08:152
269294099134,6cyclictest2728668-21/usr/sbin/munin09:03:132
269294099133,7cyclictest2838190-21ssh11:28:122
269294099132,6cyclictest2725115-21cut08:53:032
269294099132,4cyclictest2802641-21ssh10:43:012
269294099131,9cyclictest2708460-21aten_r2power_po08:03:002
269293999134,7cyclictest2698454-21/usr/sbin/munin07:33:111
269293999133,5cyclictest2705124-21/usr/sbin/munin07:53:011
269293999131,9cyclictest2708362-21aten_r2power_cu08:03:001
269293999131,10cyclictest2726745-21tune2fs08:58:011
269293999131,10cyclictest2721784-21/usr/sbin/munin08:43:021
269293999130,11cyclictest2695543-21cut07:23:031
269293899133,8cyclictest2696411-21grep07:23:130
112699130,3rtkit-daemon0-21swapper/207:13:452
269294199124,6cyclictest0-21swapper/310:33:113
269294199124,5cyclictest0-21swapper/312:08:053
269294199124,5cyclictest0-21swapper/307:16:043
269294199123,6cyclictest0-21swapper/309:33:173
269294199123,4cyclictest0-21swapper/310:53:013
269294199123,4cyclictest0-21swapper/309:53:013
269294199122,6cyclictest0-21swapper/310:58:013
269294199122,6cyclictest0-21swapper/310:43:003
269294199122,5cyclictest0-21swapper/311:43:053
269294199122,4cyclictest0-21swapper/307:42:273
269294199121,7cyclictest0-21swapper/311:33:073
269294199121,7cyclictest0-21swapper/307:33:093
269294199121,6cyclictest0-21swapper/308:13:193
269294099128,3cyclictest762-21in:imuxsock11:31:082
269294099125,4cyclictest2705301-21/usr/sbin/munin07:53:022
269294099124,5cyclictest2732065-21/usr/sbin/munin09:13:032
269294099123,8cyclictest2706944-21/usr/sbin/munin07:57:592
269293999125,5cyclictest0-21swapper/111:58:081
269293999122,6cyclictest2711996-21tune2fs08:13:091
269293999121,6cyclictest2697084-21/usr/sbin/munin07:28:011
269294199114,3cyclictest0-21swapper/310:23:013
269294199112,6cyclictest0-21swapper/312:03:073
269294199112,5cyclictest0-21swapper/309:08:023
269294199111,6cyclictest0-21swapper/310:31:043
269294199111,6cyclictest0-21swapper/308:38:033
269294199111,5cyclictest0-21swapper/311:03:033
269294199111,5cyclictest0-21swapper/309:47:333
269294099111,6cyclictest2695186-21aten_r2power_cu07:23:012
269293999112,8cyclictest2702073-21grep07:43:041
269294199104,5cyclictest0-21swapper/312:43:083
269294199104,3cyclictest0-21swapper/307:38:043
269294199103,4cyclictest0-21swapper/310:48:083
269294199103,3cyclictest0-21swapper/309:18:023
269294199102,6cyclictest0-21swapper/309:03:113
269294199102,4cyclictest0-21swapper/312:31:213
269294199101,5cyclictest0-21swapper/310:38:033
269294199101,5cyclictest0-21swapper/308:48:063
269294199101,5cyclictest0-21swapper/308:18:033
269294199101,4cyclictest0-21swapper/311:46:193
26929419994,4cyclictest0-21swapper/310:03:013
26929419994,3cyclictest0-21swapper/308:08:023
26929419994,3cyclictest0-21swapper/308:03:073
26929419993,4cyclictest0-21swapper/311:57:583
26929419993,4cyclictest0-21swapper/308:53:053
26929419993,3cyclictest0-21swapper/309:38:043
26929419992,6cyclictest0-21swapper/309:58:033
26929419992,5cyclictest0-21swapper/307:26:193
26929419992,3cyclictest0-21swapper/307:57:593
26929419991,6cyclictest0-21swapper/312:18:013
26929419990,6cyclictest0-21swapper/308:23:033
26929419990,4cyclictest0-21swapper/311:26:303
26929419990,4cyclictest0-21swapper/307:51:353
26929419962,2cyclictest0-21swapper/307:16:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional