You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 17:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack2slot7.osadl.org (updated Thu Dec 04, 2025 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293358299413,24cyclictest0-21swapper/012:38:100
2933583993425,3cyclictest111788-21systemd-journal10:50:201
293358299331,31cyclictest0-21swapper/010:06:260
293358299321,28cyclictest3092830-21ssh11:43:120
2933582993121,4cyclictest0-21swapper/011:48:180
293358299280,26cyclictest2974980-21ssh09:18:050
293358299270,3cyclictest2993976-21uniq09:43:060
2933582992621,3cyclictest0-21swapper/010:51:570
2933582992615,1cyclictest0-21swapper/009:58:120
2933582992612,3cyclictest2990671-21/usr/sbin/munin09:38:140
2933583992520,4cyclictest3037789-21cut10:38:081
2933582992518,2cyclictest0-21swapper/012:46:090
293358299251,18cyclictest0-21swapper/009:13:110
293358399248,5cyclictest11050irq/107-eth%d10:33:111
293358399247,7cyclictest171rcu_preempt07:16:091
293358299245,18cyclictest3038269-21ntpq10:38:110
2933582992418,1cyclictest0-21swapper/009:28:200
293358299241,22cyclictest0-21swapper/012:23:090
293358299241,19cyclictest0-21swapper/012:13:050
293358299241,16cyclictest3042385-21sed10:43:080
293358499234,17cyclictest605-21avahi-daemon11:38:102
293358499234,17cyclictest605-21avahi-daemon11:38:102
293358299234,18cyclictest762-21in:imuxsock10:48:310
293358299233,16cyclictest122792-21ntpd09:48:140
293358299233,16cyclictest122792-21ntpd09:48:140
293358499224,13cyclictest3006392-21ssh09:58:122
293358499223,14cyclictest3092830-21ssh11:43:122
2933584992215,4cyclictest0-21swapper/209:43:062
293358299225,16cyclictest764-21rs:main0
293358299220,16cyclictest3074784-21/usr/sbin/munin11:23:080
293358499215,10cyclictest3063663-21ntp_states11:08:142
293358499211,18cyclictest2989665-21/usr/sbin/munin09:38:102
293358399214,13cyclictest3014042-21/usr/sbin/munin10:08:111
293358399213,14cyclictest111788-21systemd-journal11:08:021
293358499202,13cyclictest3034837-21ssh10:33:272
2933584992016,3cyclictest0-21swapper/209:33:382
2933584992013,5cyclictest259-21jbd2/mmcblk0p2-808:33:442
2933584992013,4cyclictest0-21swapper/209:28:092
293358299203,11cyclictest2933533-21latency_hist07:16:090
2933582992012,6cyclictest2977862-1kworker/0:1H+kblockd11:08:020
293358299201,18cyclictest3030425-21proc_pri10:28:140
293358299201,18cyclictest3002164-21awk09:53:100
293358499192,13cyclictest3040907-21ssh10:41:112
293358499192,12cyclictest2933789-21latency_hist07:16:092
293358399195,2cyclictest3021200-21ssh10:18:041
293358399192,14cyclictest2972855-21ntpq09:13:111
2933583991913,4cyclictest3104989-21fschecks_count11:58:051
293358299193,3cyclictest3087900-21gltestperf11:38:070
293358299193,3cyclictest3087900-21gltestperf11:38:070
293358299193,14cyclictest3129445-21sendmail-msp12:33:060
293358299192,5cyclictest2958785-21latency_hist08:31:090
293358299192,14cyclictest3131527-21missed_timers12:28:110
2933582991913,4cyclictest3122503-21ssh12:18:080
2933582991913,4cyclictest3122503-21ssh12:18:080
2933582991913,4cyclictest2963589-21sendmail-msp08:53:110
2933582991913,1cyclictest0-21swapper/012:11:090
2933582991913,1cyclictest0-21swapper/012:11:090
293358299191,18cyclictest0-21swapper/008:13:030
293358299191,18cyclictest0-21swapper/008:13:030
293358499186,8cyclictest111788-21systemd-journal12:29:262
293358499183,10cyclictest619-21mta-sts-daemon11:23:122
293358499182,13cyclictest3079984-21ntp_states11:28:122
293358399189,6cyclictest2969097-21fschecks_time09:03:051
293358399189,6cyclictest0-21swapper/110:53:101
293358399183,8cyclictest2986161-21aten_r2power_fr09:33:051
293358399183,6cyclictest2997885-21fschecks_time09:48:081
293358399183,6cyclictest2997885-21fschecks_time09:48:081
293358399183,10cyclictest1005-21mta-sts-daemon07:51:111
2933583991813,3cyclictest3006371-21ssh09:58:121
2933583991812,2cyclictest0-21swapper/108:33:121
293358299186,1cyclictest2986599-21ssh09:33:090
293358299184,13cyclictest0-21swapper/008:48:020
293358299184,10cyclictest3077818-21sendmail-msp11:30:540
293358299183,13cyclictest1005-21mta-sts-daemon08:43:110
293358299183,12cyclictest151ktimers/012:03:040
293358299182,5cyclictest3054628-21df10:58:070
2933582991818,0cyclictest0-21swapper/011:53:120
112699180,4rtkit-daemon1125-21rtkit-daemon09:21:392
112699180,3rtkit-daemon1125-21rtkit-daemon11:01:460
293358499174,3cyclictest0-21swapper/210:18:042
293358499172,14cyclictest3013233-21/usr/sbin/munin10:08:112
2933584991713,2cyclictest1005-21mta-sts-daemon10:23:122
2933584991712,4cyclictest0-21swapper/211:53:112
2933584991712,4cyclictest0-21swapper/208:13:082
2933584991712,4cyclictest0-21swapper/208:13:082
2933584991711,5cyclictest762-21in:imuxsock08:41:082
293358499171,15cyclictest2968449-21grep08:58:282
293358399174,8cyclictest111788-21systemd-journal12:28:051
293358399174,10cyclictest762-21in:imuxsock12:07:361
293358399174,10cyclictest762-21in:imuxsock12:07:361
293358399172,8cyclictest3111915-21sendmail-msp12:13:051
293358399172,12cyclictest259-21jbd2/mmcblk0p2-811:30:541
293358399172,10cyclictest3054404-21sh10:58:061
2933583991715,1cyclictest0-21swapper/109:53:041
2933583991714,2cyclictest3145459-21unixbench_multi12:43:141
293358399171,13cyclictest2975726-21/usr/sbin/munin09:18:131
293358299179,3cyclictest762-21in:imuxsock10:33:110
293358299173,11cyclictest111788-21systemd-journal10:13:130
293358299172,11cyclictest3010135-21ssh10:03:080
293358299171,13cyclictest55022-1kworker/0:0H+mmc_complete07:51:110
2933582991710,3cyclictest0-21swapper/009:08:250
293358299170,7cyclictest2981650-21systemd09:26:090
112699170,3rtkit-daemon1008-21mta-sts-daemon10:26:252
293358599162,6cyclictest0-21swapper/309:14:163
293358499165,5cyclictest2970145-21ls09:06:092
293358499164,9cyclictest2943472-21sendmail-msp07:51:422
293358499163,4cyclictest0-21swapper/212:18:052
293358499163,4cyclictest0-21swapper/212:18:052
293358499163,11cyclictest2940415-21/usr/sbin/munin07:38:022
293358399164,10cyclictest2983456-21needreboot09:28:111
293358399163,9cyclictest3094936-21sendmail-msp11:48:171
293358399162,11cyclictest2949436-21grep08:03:121
293358399162,11cyclictest111788-21systemd-journal12:33:131
293358399161,9cyclictest3125027-21latency_hist12:21:091
293358399161,9cyclictest3125027-21latency_hist12:21:091
2933583991613,2cyclictest2944122-21gltestperf07:48:051
2933583991613,2cyclictest1006-21mta-sts-daemon11:03:101
2933583991612,3cyclictest2962931-21ntp_kernel_err08:43:101
293358399161,12cyclictest2996332-21munin-run09:46:081
293358399161,10cyclictest3090677-21cat11:41:091
293358399161,10cyclictest3090677-21cat11:41:091
293358399160,3cyclictest11050irq/107-eth%d11:43:071
293358299169,1cyclictest3083503-21ssh11:33:070
293358299168,1cyclictest0-21swapper/010:17:130
293358299165,8cyclictest2968062-21/usr/sbin/munin08:58:240
293358299163,11cyclictest2942966-21/usr/sbin/munin07:43:140
293358299162,10cyclictest3105850-21/usr/sbin/munin11:58:140
293358299162,10cyclictest1006-21mta-sts-daemon07:38:090
293358299161,4cyclictest1005-21mta-sts-daemon08:08:110
2933582991614,1cyclictest0-21swapper/008:03:020
293358599153,6cyclictest0-21swapper/310:48:113
293358499154,8cyclictest2957505-21fschecks_count08:28:042
293358499154,3cyclictest762-21in:imuxsock12:13:112
293358499152,9cyclictest3002362-21sh09:53:122
293358499152,8cyclictest3113888-21ntpq12:08:112
293358499152,8cyclictest3113888-21ntpq12:08:112
293358499152,11cyclictest3140771-21/usr/sbin/munin12:38:122
2933584991512,2cyclictest3099230-21latency_hist11:51:092
2933584991512,2cyclictest0-21swapper/211:03:052
2933584991512,1cyclictest0-21swapper/210:13:132
293358499151,13cyclictest2940153-21cron07:36:082
2933584991510,4cyclictest0-21swapper/207:31:082
293358499150,11cyclictest2945168-21base6407:51:092
293358399155,6cyclictest3030265-21sh10:28:111
293358399154,7cyclictest111788-21systemd-journal10:26:081
293358399153,9cyclictest3041923-21/usr/sbin/munin10:43:021
293358399153,8cyclictest3100814-21fschecks_time11:53:061
293358399153,8cyclictest2964136-21fschecks_count08:48:061
2933583991512,1cyclictest0-21swapper/109:23:161
2933583991511,2cyclictest0-21swapper/112:23:041
293358399151,10cyclictest2960891-21tune2fs08:38:051
293358299154,8cyclictest2955914-21grep08:23:060
293358299153,7cyclictest111788-21systemd-journal08:21:080
293358299152,8cyclictest2940165-21idleruntime-cro07:36:090
293358299150,11cyclictest3071554-21/usr/sbin/munin11:18:170
293358599144,6cyclictest0-21swapper/311:43:123
293358599143,5cyclictest0-21swapper/309:48:143
293358599143,5cyclictest0-21swapper/309:48:143
293358599142,7cyclictest0-21swapper/309:36:083
293358599142,5cyclictest0-21swapper/311:23:123
293358499149,3cyclictest0-21swapper/208:43:112
293358499144,6cyclictest3127597-21/usr/sbin/munin12:23:152
293358499143,9cyclictest2950965-21fschecks_time08:08:072
293358499143,6cyclictest2942031-21/usr/sbin/munin07:43:052
293358499143,4cyclictest111788-21systemd-journal11:32:482
293358499142,8cyclictest764-21rs:main2
293358499142,11cyclictest762-21in:imuxsock12:33:062
293358499142,10cyclictest3109110-21/usr/sbin/munin12:03:052
293358499142,10cyclictest3105670-21/usr/sbin/munin11:58:112
293358499141,9cyclictest2948885-21latency_hist08:01:092
2933584991412,1cyclictest0-21swapper/209:48:082
2933584991412,1cyclictest0-21swapper/209:48:082
2933584991411,2cyclictest1007-21mta-sts-daemon10:37:592
293358399144,8cyclictest3075259-21/usr/sbin/munin11:23:091
293358399144,6cyclictest3140523-21ssh12:38:091
293358399143,9cyclictest111788-21systemd-journal11:13:161
293358399143,6cyclictest2933802-21/usr/sbin/munin07:18:081
293358399142,9cyclictest3108868-21/usr/sbin/munin12:03:051
293358399141,9cyclictest3070915-21ssh11:18:081
293358399141,8cyclictest2954747-21/usr/sbin/munin08:18:111
293358399141,11cyclictest2990206-21if_end009:38:091
293358399141,10cyclictest2955890-21uniq08:23:061
2933583991410,2cyclictest0-21swapper/107:38:031
2933582991411,2cyclictest3025819-21kernelversion10:23:090
2933582991411,2cyclictest2936753-21unixbench_multi07:23:170
2933582991411,2cyclictest2936753-21unixbench_multi07:23:170
2933582991411,2cyclictest1008-21mta-sts-daemon07:18:070
293358599134,4cyclictest0-21swapper/310:53:103
293358599133,6cyclictest0-21swapper/311:53:063
293358599133,5cyclictest0-21swapper/312:13:063
293358599132,8cyclictest0-21swapper/309:18:113
293358599132,6cyclictest0-21swapper/309:43:083
293358599131,6cyclictest0-21swapper/311:38:103
293358599131,6cyclictest0-21swapper/311:38:103
293358599131,6cyclictest0-21swapper/310:58:183
293358599130,4cyclictest0-21swapper/307:51:113
293358499136,3cyclictest3054887-21/usr/sbin/munin10:58:092
293358499134,8cyclictest2954887-21/usr/sbin/munin08:18:132
293358499134,5cyclictest0-21swapper/211:12:002
293358499132,8cyclictest2935617-21tail07:23:062
293358499132,8cyclictest2935617-21tail07:23:062
293358499132,7cyclictest2957017-21munin-run08:26:092
293358499131,9cyclictest3050827-21/usr/sbin/munin10:53:082
293358499131,7cyclictest3144384-21fschecks_count12:43:052
2933584991311,2cyclictest1008-21mta-sts-daemon10:05:552
293358399136,3cyclictest0-21swapper/107:33:171
293358399134,7cyclictest3082435-21ssh11:31:261
293358399134,7cyclictest3009585-21fschecks_count10:03:041
293358399134,7cyclictest2966120-21/usr/sbin/munin08:53:091
293358399133,7cyclictest3018076-21/usr/sbin/munin10:13:111
293358399133,6cyclictest2947855-21/usr/sbin/munin07:58:061
293358399132,7cyclictest2935986-21/usr/sbin/munin07:23:081
293358399132,7cyclictest2935986-21/usr/sbin/munin07:23:081
293358399132,6cyclictest2950793-21aten_r2power_vo08:08:051
293358399131,9cyclictest2967257-21aten2.4-expect08:58:161
293358299139,2cyclictest0-21swapper/007:28:090
293358299134,6cyclictest2969656-21/usr/sbin/munin09:03:110
293358299130,8cyclictest2943837-21awk07:48:020
112699130,2rtkit-daemon1007-21mta-sts-daemon09:18:532
293358599122,6cyclictest0-21swapper/312:33:063
293358599122,6cyclictest0-21swapper/308:53:063
293358599122,6cyclictest0-21swapper/308:38:093
293358599122,5cyclictest0-21swapper/311:58:043
293358599122,5cyclictest0-21swapper/309:03:143
293358599122,5cyclictest0-21swapper/307:33:103
293358599122,4cyclictest0-21swapper/312:03:033
293358599121,8cyclictest0-21swapper/312:29:423
293358599121,7cyclictest0-21swapper/310:13:173
293358599121,6cyclictest0-21swapper/308:23:113
293358599121,5cyclictest0-21swapper/310:41:113
293358499128,3cyclictest0-21swapper/210:48:112
293358499122,4cyclictest3071756-21/usr/sbin/munin11:18:212
293358499121,9cyclictest2965882-21/usr/sbin/munin08:53:082
293358499121,7cyclictest2972036-21/usr/sbin/munin09:13:082
293358499121,7cyclictest2970168-21sendmail-msp09:08:252
293358499121,2cyclictest607-21dbus-daemon07:21:092
293358399121,9cyclictest2942192-21aten_r2power_vo07:43:071
293358399121,9cyclictest2938438-21unixbench_multi07:28:171
293358399121,8cyclictest762-21in:imuxsock09:08:191
293358399121,8cyclictest2958630-21cron08:31:081
293358299123,8cyclictest2948104-21/usr/sbin/munin07:58:080
293358299120,8cyclictest2958862-21/usr/sbin/munin08:33:080
293358299120,7cyclictest0-21swapper/008:38:020
293358599115,3cyclictest0-21swapper/307:28:113
293358599114,4cyclictest0-21swapper/308:08:113
293358599113,5cyclictest0-21swapper/309:08:123
293358599113,4cyclictest0-21swapper/311:08:023
293358599113,4cyclictest0-21swapper/310:03:083
293358599113,4cyclictest0-21swapper/308:13:123
293358599113,4cyclictest0-21swapper/308:13:123
293358599112,7cyclictest0-21swapper/311:28:113
293358599112,7cyclictest0-21swapper/309:28:063
293358599112,6cyclictest0-21swapper/311:48:083
293358599112,5cyclictest0-21swapper/312:23:103
293358599112,5cyclictest0-21swapper/311:16:093
293358599112,3cyclictest0-21swapper/310:21:173
293358599111,7cyclictest0-21swapper/310:21:093
293358599111,6cyclictest0-21swapper/309:53:163
293358499113,6cyclictest2949802-21/usr/sbin/munin08:03:162
293358399113,5cyclictest2950408-21sendmail-msp08:13:041
293358399113,5cyclictest2950408-21sendmail-msp08:13:041
293358599103,5cyclictest0-21swapper/311:36:093
293358599103,4cyclictest0-21swapper/311:18:223
293358599103,4cyclictest0-21swapper/309:56:123
293358599102,7cyclictest0-21swapper/307:16:093
293358599102,6cyclictest0-21swapper/309:38:073
293358599102,5cyclictest0-21swapper/312:18:433
293358599102,5cyclictest0-21swapper/312:18:433
293358599102,5cyclictest0-21swapper/308:58:213
293358599102,5cyclictest0-21swapper/308:03:133
293358599102,5cyclictest0-21swapper/307:38:063
293358599102,5cyclictest0-21swapper/307:23:043
293358599102,5cyclictest0-21swapper/307:23:043
293358599102,4cyclictest0-21swapper/310:08:113
293358599102,4cyclictest0-21swapper/307:47:533
293358599102,3cyclictest0-21swapper/312:06:363
293358599102,3cyclictest0-21swapper/312:06:363
293358599101,4cyclictest0-21swapper/310:36:593
29335859995,3cyclictest0-21swapper/310:33:073
29335859994,3cyclictest0-21swapper/310:27:593
29335859994,3cyclictest0-21swapper/308:43:113
29335859993,4cyclictest0-21swapper/307:57:373
29335859992,4cyclictest0-21swapper/307:43:053
29335859992,3cyclictest0-21swapper/308:33:123
29335859991,6cyclictest0-21swapper/308:18:073
29335859991,5cyclictest0-21swapper/309:23:103
29335859991,5cyclictest0-21swapper/308:48:143
29335859984,2cyclictest0-21swapper/312:38:123
29335859982,5cyclictest0-21swapper/311:03:043
29335859982,3cyclictest0-21swapper/307:18:113
29335859980,5cyclictest0-21swapper/312:43:033
29335859980,5cyclictest0-21swapper/308:28:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional