You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 17:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7.osadl.org (updated Sat Dec 06, 2025 12:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2979436996113,45cyclictest0-21swapper/009:26:240
2979436995422,29cyclictest3134827-21sh11:33:140
2979436995227,5cyclictest0-21swapper/010:08:050
297943699477,38cyclictest3185402-21sendmail-msp12:34:480
297943699473,11cyclictest3111080-21sendmail-msp11:09:320
297943699466,27cyclictest0-21swapper/009:36:140
2979436994628,9cyclictest3116622-21/usr/sbin/munin11:13:120
2979436994616,29cyclictest0-21swapper/012:10:360
2979437994127,11cyclictest3114100-21ssh11:08:471
2979436994119,18cyclictest3155181-21latency_hist11:56:110
297943699398,24cyclictest3018795-21ntpq09:13:140
2979436993922,15cyclictest3128045-21sendmail-msp11:28:320
2979436993922,15cyclictest3128045-21sendmail-msp11:28:320
2979437993729,6cyclictest2979433-21cyclictest12:13:161
297943699362,7cyclictest3198319-21ntp_states12:38:280
2979436993617,3cyclictest3185402-21sendmail-msp12:26:120
297943699357,10cyclictest619-21mta-sts-daemon09:53:120
297943699354,25cyclictest3044032-21sendmail-msp09:46:180
297943699353,16cyclictest0-21swapper/011:51:110
2979436993517,14cyclictest3078002-21sendmail-msp10:26:200
297943699341,7cyclictest2993672-21ntpq07:58:120
2979436993417,16cyclictest764-21rs:main0
297943699341,29cyclictest0-21swapper/010:50:220
297943699340,10cyclictest3175131-21latency_hist12:16:110
297943799332,15cyclictest3145855-21sendmail-msp11:55:421
297943699332,6cyclictest3012211-21ld08:53:250
297943699325,3cyclictest2995908-21sendmail-msp08:06:160
297943799312,11cyclictest3167391-21ssh12:08:131
297943699315,2cyclictest0-21swapper/010:32:550
297943699312,15cyclictest3108017-21/usr/sbin/munin11:03:080
297943699303,7cyclictest3183048-21/usr/sbin/munin12:23:150
2979436993018,7cyclictest0-21swapper/011:41:340
2979437992911,4cyclictest11050irq/107-eth%d11:33:141
297943699297,18cyclictest0-21swapper/010:03:120
297943699291,9cyclictest3041335-21grep09:43:070
297943699291,2cyclictest3092303-21uname10:43:110
2979438992821,6cyclictest3059652-21ssh10:03:352
2979437992811,14cyclictest3035534-21sh09:36:111
297943699285,6cyclictest3009783-21/usr/sbin/munin08:48:080
297943699284,6cyclictest3087602-21/usr/sbin/munin10:38:050
297943699281,5cyclictest3000721-21unixbench_multi08:18:190
297943899272,6cyclictest171rcu_preempt09:36:112
297943699276,18cyclictest2979433-21cyclictest07:16:100
297943699271,9cyclictest3128015-21run-parts11:26:100
297943899265,14cyclictest3134017-21ssh11:33:052
297943799262,4cyclictest11050irq/107-eth%d11:46:121
2979437992619,4cyclictest0-21swapper/109:43:131
297943699265,20cyclictest0-21swapper/008:33:070
297943699262,7cyclictest2995865-21cron08:06:100
297943699261,8cyclictest3177029-21cut12:18:060
297943699261,6cyclictest2991708-21/usr/sbin/munin07:53:090
297943699260,7cyclictest3070761-21cpuspeed10:18:050
297943899256,13cyclictest3078002-21sendmail-msp10:31:172
297943799255,15cyclictest111788-21systemd-journal09:52:331
297943799251,9cyclictest0-21swapper/110:26:521
297943699257,3cyclictest3156833-21gltestperf11:58:070
297943699252,8cyclictest3162307-21rm12:03:130
297943699252,6cyclictest2988330-21grep07:43:060
297943699252,11cyclictest3076385-21grep10:23:170
2979436992510,2cyclictest3102589-21munin-run10:56:100
297943899245,17cyclictest967-21snmpd08:47:072
297943899242,17cyclictest3132527-21latency_hist11:31:112
297943899242,17cyclictest3132527-21latency_hist11:31:112
2979438992418,3cyclictest3079497-21fschecks_time10:28:112
297943799245,14cyclictest2243333-21munin-node07:16:111
297943799242,9cyclictest3116622-21/usr/sbin/munin11:13:101
297943699245,5cyclictest3201157-21sh12:41:410
297943699241,6cyclictest3124037-21latency_hist11:21:100
297943699241,11cyclictest2999076-21grep08:13:180
297943899236,11cyclictest3146100-21ssh11:46:152
297943899235,14cyclictest3046295-21/usr/sbin/munin09:48:152
297943899233,13cyclictest3069754-21sh10:16:172
2979437992315,5cyclictest111788-21systemd-journal10:34:051
297943699234,5cyclictest3054049-21/usr/sbin/munin09:58:060
297943699232,7cyclictest2986499-21/usr/sbin/munin07:38:070
297943699232,5cyclictest3027465-21grep09:26:110
2979436992318,4cyclictest0-21swapper/007:46:300
2979436992311,10cyclictest0-21swapper/009:18:170
297943699231,10cyclictest3138246-21ssh11:38:080
297943699230,23cyclictest0-21swapper/008:58:050
297943699230,23cyclictest0-21swapper/008:58:050
297943899227,12cyclictest967-21snmpd11:14:022
297943899224,12cyclictest3012211-21ld08:53:252
2979438992218,3cyclictest3008365-21cut08:43:072
2979438992214,6cyclictest0-21swapper/207:16:112
297943799224,13cyclictest111788-21systemd-journal12:34:481
297943799222,14cyclictest3060979-21sendmail-msp10:13:511
297943799222,14cyclictest3060979-21sendmail-msp10:13:511
297943699223,5cyclictest0-21swapper/010:58:200
297943699222,7cyclictest3002194-21/usr/sbin/munin08:23:160
297943699221,8cyclictest2982710-21idleruntime-cro07:26:100
297943699221,20cyclictest3017734-21latency_hist09:11:110
297943699220,9cyclictest111788-21systemd-journal07:28:110
297943899215,13cyclictest111788-21systemd-journal10:51:112
297943899214,9cyclictest11050irq/107-eth%d09:58:122
297943899214,13cyclictest111788-21systemd-journal09:28:132
297943899211,17cyclictest3052502-21ssh09:56:062
297943799215,13cyclictest3019298-21diskmemload09:46:161
297943799213,17cyclictest3104704-21ntpq10:58:151
2979437992116,1cyclictest11050irq/107-eth%d11:58:111
2979437992112,6cyclictest1006-21mta-sts-daemon08:41:131
2979437992111,5cyclictest3094583-21sendmail-msp10:52:441
297943699218,2cyclictest3015561-21find09:03:140
297943699211,7cyclictest3060979-21sendmail-msp10:13:190
297943699211,7cyclictest3060979-21sendmail-msp10:13:190
297943699211,4cyclictest2979802-21aten_r2power_vo07:18:050
297943699211,2cyclictest3009125-21find08:43:160
297943899206,9cyclictest1005-21mta-sts-daemon09:21:182
297943899204,8cyclictest3060979-21sendmail-msp10:11:122
297943899204,8cyclictest3060979-21sendmail-msp10:11:122
297943899202,12cyclictest3041973-21/usr/sbin/munin09:43:132
2979438992014,4cyclictest0-21swapper/211:06:122
297943799203,9cyclictest111788-21systemd-journal08:50:361
2979437992011,4cyclictest111788-21systemd-journal11:26:391
2979437992011,4cyclictest111788-21systemd-journal11:26:391
2979437992010,7cyclictest0-21swapper/110:45:111
297943999193,10cyclictest0-21swapper/311:33:143
297943899199,9cyclictest259-21jbd2/mmcblk0p2-812:12:472
297943899194,11cyclictest3192987-21/usr/sbin/munin12:33:162
2979438991912,4cyclictest0-21swapper/212:23:152
297943899191,16cyclictest3187979-21ssh12:28:162
297943799192,11cyclictest11050irq/107-eth%d11:18:131
297943799192,11cyclictest11050irq/107-eth%d10:50:541
2979437991915,3cyclictest3024698-21gltestperf09:23:071
2979437991913,5cyclictest3111066-21idleruntime-cro11:06:101
297943699192,5cyclictest3002745-21/usr/sbin/munin08:28:050
297943999184,9cyclictest0-21swapper/309:48:153
297943899185,8cyclictest0-21swapper/210:08:052
297943899185,7cyclictest3143407-21ssh11:43:122
297943899184,9cyclictest259-21jbd2/mmcblk0p2-811:53:432
2979438991814,3cyclictest3197723-21cut12:38:232
2979438991814,3cyclictest3157231-21ntp_kernel_pll_11:58:112
297943799183,9cyclictest3016024-21sendmail-msp09:13:141
297943799182,14cyclictest3071122-21/usr/sbin/munin10:18:071
2979437991811,6cyclictest0-21swapper/109:58:121
297943799180,17cyclictest3185622-21ssh12:26:121
297943999173,9cyclictest0-21swapper/310:31:163
297943999173,6cyclictest0-21swapper/310:46:133
297943899175,7cyclictest2990523-21/usr/sbin/munin07:48:132
297943899172,13cyclictest2999188-21idleruntime-cro08:16:102
297943899172,10cyclictest111788-21systemd-journal09:13:142
2979438991716,0cyclictest3165741-21ssh12:07:012
2979438991715,2cyclictest1007-21mta-sts-daemon07:23:102
2979438991714,2cyclictest3014282-21unixbench_singl08:58:182
2979438991714,2cyclictest3014282-21unixbench_singl08:58:182
297943799179,7cyclictest607-21dbus-daemon09:01:101
297943799179,7cyclictest607-21dbus-daemon09:01:101
297943799172,8cyclictest2995874-21latency_hist08:06:101
297943799172,12cyclictest111788-21systemd-journal08:55:361
297943799171,4cyclictest3075646-21ssh10:23:091
2979437991713,2cyclictest1007-21mta-sts-daemon12:18:101
112699170,3rtkit-daemon1125-21rtkit-daemon12:45:181
297943999163,8cyclictest0-21swapper/312:26:123
297943999163,6cyclictest0-21swapper/311:48:083
297943899167,8cyclictest967-21snmpd11:39:242
297943899165,10cyclictest3120029-21ssh11:16:292
297943899164,8cyclictest111788-21systemd-journal08:06:162
297943899163,9cyclictest3125130-21munin-node11:23:042
297943899162,13cyclictest605-21avahi-daemon09:18:092
2979438991612,4cyclictest2993655-21ntp_kernel_pll_07:58:122
297943799164,8cyclictest111788-21systemd-journal07:53:111
297943799163,10cyclictest3021445-21ssh09:18:121
297943799162,10cyclictest2982745-21sendmail-msp07:33:121
297943699160,13cyclictest3007579-21grep08:41:110
297943999153,8cyclictest0-21swapper/311:31:113
297943999153,8cyclictest0-21swapper/311:31:113
297943999152,8cyclictest0-21swapper/309:43:133
297943999152,7cyclictest0-21swapper/312:06:283
297943999152,5cyclictest0-21swapper/309:13:123
297943899154,7cyclictest3086808-21ssh10:36:442
297943899154,7cyclictest3002505-21sendmail-msp08:26:182
297943899153,8cyclictest3094583-21sendmail-msp10:46:152
297943899151,4cyclictest3092563-21ntp_kernel_pll_10:43:132
2979438991511,3cyclictest3032621-21grep09:33:132
297943799154,8cyclictest3087535-21aten_r2power_cu10:38:051
297943799154,6cyclictest11050irq/107-eth%d07:43:101
297943799153,9cyclictest3029464-21ntpq09:28:131
297943799153,8cyclictest3007411-21munin-run08:41:101
297943799153,7cyclictest111788-21systemd-journal10:08:201
2979437991512,2cyclictest1005-21mta-sts-daemon07:18:081
2979437991511,3cyclictest2994176-21grep07:58:181
2979437991510,3cyclictest0-21swapper/109:33:181
2979436991512,2cyclictest2985070-21gltestperf07:33:090
297943999145,5cyclictest0-21swapper/311:53:113
297943999145,5cyclictest0-21swapper/311:08:153
297943999144,6cyclictest0-21swapper/310:18:063
297943999143,8cyclictest0-21swapper/307:16:113
297943999143,6cyclictest0-21swapper/309:28:133
297943999143,5cyclictest0-21swapper/308:41:133
297943899148,5cyclictest2243333-21munin-node11:02:332
297943899144,6cyclictest3074703-21sh10:22:442
297943799143,8cyclictest0-21swapper/112:23:151
297943799142,9cyclictest3163082-21ssh12:03:351
297943799142,8cyclictest3142542-21/usr/sbin/munin11:43:051
297943799142,11cyclictest3138774-21/usr/sbin/munin11:38:111
2979437991410,3cyclictest1125-21rtkit-daemon07:31:111
297943999134,5cyclictest0-21swapper/308:48:083
297943999134,4cyclictest0-21swapper/309:36:113
297943999133,5cyclictest0-21swapper/312:18:123
297943999133,5cyclictest0-21swapper/309:53:123
297943999133,5cyclictest0-21swapper/307:53:073
297943999132,7cyclictest0-21swapper/312:38:283
297943999132,7cyclictest0-21swapper/312:16:113
297943999132,6cyclictest0-21swapper/308:18:063
297943899135,6cyclictest0-21swapper/209:08:062
297943899135,5cyclictest3104872-21/usr/sbin/munin10:58:172
297943899134,6cyclictest3176982-21/usr/sbin/munin12:18:062
297943899133,8cyclictest3203663-21grep12:43:312
297943899133,8cyclictest2980387-21/usr/sbin/munin07:18:092
297943899133,7cyclictest3162372-21ssh12:03:142
297943899133,7cyclictest3007069-21/usr/sbin/munin08:38:152
297943899132,9cyclictest2991512-21grep07:53:082
2979438991311,1cyclictest0-21swapper/207:46:112
2979438991311,1cyclictest0-21swapper/207:28:112
297943799136,5cyclictest1008-21mta-sts-daemon09:11:111
297943799134,6cyclictest0-21swapper/107:38:141
297943799133,8cyclictest3003502-21/usr/sbin/munin08:28:121
297943799133,7cyclictest3014806-21/usr/sbin/munin09:03:071
297943799132,7cyclictest3198427-21/usr/sbin/munin12:38:291
297943799132,7cyclictest3001861-21run-parts08:23:101
297943799131,9cyclictest3125489-21fschecks_time11:23:071
297943799131,8cyclictest2999780-21fschecks_time08:18:081
2979437991310,1cyclictest0-21swapper/110:03:181
297943999125,4cyclictest0-21swapper/310:26:203
297943999123,6cyclictest0-21swapper/312:32:203
297943999123,5cyclictest0-21swapper/311:13:123
297943999123,5cyclictest0-21swapper/309:33:363
297943999123,5cyclictest0-21swapper/309:06:103
297943999123,4cyclictest0-21swapper/307:58:123
297943999122,6cyclictest0-21swapper/309:57:103
297943999122,3cyclictest0-21swapper/312:21:263
297943999120,7cyclictest0-21swapper/310:08:053
297943999120,6cyclictest0-21swapper/312:43:313
297943899124,6cyclictest0-21swapper/208:03:052
297943899122,8cyclictest3014577-21/usr/sbin/munin09:03:072
297943899121,8cyclictest3001067-21/usr/sbin/munin08:23:042
297943799128,3cyclictest2996562-21gltestperf08:08:081
297943799121,8cyclictest0-21swapper/107:48:131
297943999114,5cyclictest0-21swapper/310:51:113
297943999114,3cyclictest0-21swapper/311:03:143
297943999113,6cyclictest0-21swapper/310:03:073
297943999113,4cyclictest0-21swapper/307:43:113
297943999112,6cyclictest0-21swapper/308:53:243
297943999112,5cyclictest0-21swapper/311:58:123
297943999112,5cyclictest0-21swapper/310:13:193
297943999112,5cyclictest0-21swapper/310:13:193
297943999112,4cyclictest0-21swapper/311:18:043
297943999112,4cyclictest0-21swapper/307:28:103
297943999111,7cyclictest0-21swapper/311:24:403
297943999111,6cyclictest0-21swapper/308:28:143
297943999111,5cyclictest0-21swapper/309:08:093
297943999111,4cyclictest0-21swapper/308:08:033
297943899113,7cyclictest0-21swapper/208:33:182
297943899112,8cyclictest2985010-21grep07:33:092
297943899112,7cyclictest0-21swapper/207:38:102
297943799118,2cyclictest762-21in:imuxsock08:13:091
297943799112,7cyclictest3004672-21grep08:33:091
297943799112,6cyclictest2981579-21grep07:23:061
297943999104,3cyclictest0-21swapper/309:18:123
297943999103,6cyclictest0-21swapper/311:41:343
297943999103,5cyclictest0-21swapper/310:23:053
297943999103,5cyclictest0-21swapper/308:58:063
297943999103,5cyclictest0-21swapper/308:58:063
297943999103,4cyclictest0-21swapper/310:58:123
297943999103,4cyclictest0-21swapper/310:38:083
297943999103,4cyclictest0-21swapper/308:38:103
297943999102,5cyclictest0-21swapper/308:03:093
297943999102,5cyclictest0-21swapper/307:23:073
297943999102,4cyclictest0-21swapper/308:33:203
297943999101,6cyclictest0-21swapper/311:41:113
297943999101,5cyclictest0-21swapper/308:13:103
297943999101,5cyclictest0-21swapper/307:33:053
297943899101,7cyclictest0-21swapper/208:18:172
29794399995,2cyclictest0-21swapper/312:03:083
29794399994,4cyclictest0-21swapper/310:43:103
29794399994,4cyclictest0-21swapper/307:38:083
29794399991,5cyclictest0-21swapper/307:46:303
29794399990,5cyclictest0-21swapper/308:26:103
29794399982,3cyclictest0-21swapper/307:18:073
29794399980,4cyclictest0-21swapper/309:21:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional