You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 05:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun May 24, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2847253993933,4cyclictest2888865-21latency_hist19:50:012
2847253993832,4cyclictest2988606-21diskstats21:25:142
2847257993732,3cyclictest3097313-21gpgv23:10:003
2847251993732,4cyclictest3081699-21latency_hist22:55:011
2847247993530,4cyclictest2983242-21/usr/sbin/munin21:20:140
2847257993428,4cyclictest335557-21systemd-journal20:47:363
2847251993317,11cyclictest2945964-21apt-get20:45:001
2847257993125,4cyclictest2889112-21/usr/sbin/munin19:50:253
2847247993125,4cyclictest335557-21systemd-journal20:37:330
2847247993025,4cyclictest2971407-21ntpq21:05:230
2847253992822,4cyclictest3119365-21cp23:30:002
2847247992822,4cyclictest3066201-21latency_hist22:40:010
2847253992720,5cyclictest3107989-21latency_hist23:20:002
2847251992722,3cyclictest2967289-21kworker/u8:2+flush-259:021:10:161
2847251992721,4cyclictest3054615-21ntpq22:25:211
2847253992622,3cyclictest2930735-21kworker/u8:3+flush-259:020:40:252
2847253992613,4cyclictest335557-21systemd-journal00:35:002
2847253992521,3cyclictest3119836-21kworker/u8:3+flush-259:000:15:182
2847253992519,4cyclictest3181624-21latency_hist00:30:012
2847247992518,5cyclictest3092037-21latency_hist23:05:000
2847253992417,5cyclictest3140301-21latency_hist23:50:012
2847251992419,3cyclictest335557-21systemd-journal19:36:051
284724799249,5cyclictest2951803-21find20:50:110
2847247992419,3cyclictest335557-21systemd-journal23:30:020
2847247992411,9cyclictest2962245-21cpuspeed_turbos21:00:120
2847257992320,2cyclictest2966707-21fwupd21:01:063
2847253992318,4cyclictest335557-21systemd-journal22:22:052
2847251992318,3cyclictest335557-21systemd-journal22:39:591
2847247992317,5cyclictest2898305-21ntpq19:55:220
2847257992218,3cyclictest3097862-21/usr/sbin/munin23:10:213
2847253992218,3cyclictest3119836-21kworker/u8:3+flush-259:000:20:172
2847253992217,3cyclictest335557-21systemd-journal22:32:552
2847253992216,4cyclictest2847169-21latency_hist19:10:012
2847251992216,4cyclictest2930501-21latency_hist20:30:001
2847257992118,2cyclictest509-21rs:main3
2847257992116,3cyclictest509-21rs:main3
2847257992116,3cyclictest335557-21systemd-journal23:20:013
2847257992116,3cyclictest335557-21systemd-journal23:20:003
2847257992115,5cyclictest3185915-21ntpq00:30:223
2847253992117,3cyclictest2934803-21ntpq20:30:202
2847251992116,4cyclictest3045139-21latency_hist22:20:011
2847251992116,3cyclictest3061059-21gpgv22:35:011
284724799219,3cyclictest2916951-21grep20:15:190
284724799218,4cyclictest2873155-21apt-config19:35:010
284724799217,4cyclictest3024460-21cat22:00:010
2847247992116,4cyclictest335557-21systemd-journal00:01:050
2847247992115,5cyclictest3175599-21ntpq00:20:230
2847257992017,2cyclictest3114284-21apt-get23:25:113
2847257992017,2cyclictest2967289-21kworker/u8:2+events_unbound21:45:203
2847257992015,4cyclictest2924956-21latency_hist20:25:003
2847257992014,4cyclictest335557-21systemd-journal23:36:053
2847257992014,4cyclictest335557-21systemd-journal20:30:053
2847257992014,4cyclictest2956716-21latency_hist20:55:003
2847253992015,3cyclictest3034759-21cron22:10:002
2847251992014,4cyclictest3086041-21ntpq22:55:221
284724799208,4cyclictest3044526-21timerandwakeup22:15:250
284724799207,5cyclictest3112882-21sensors23:20:230
284724799207,5cyclictest3112882-21sensors23:20:230
284724799207,4cyclictest2959298-21cut20:55:170
2847247992014,4cyclictest2946299-21base6420:45:000
2847257991916,2cyclictest2862979-21/usr/sbin/munin19:25:233
2847257991915,3cyclictest3150892-21kworker/u8:1+flush-259:000:25:173
2847257991913,4cyclictest2998530-21latency_hist21:35:013
2847253991913,4cyclictest335557-21systemd-journal22:27:062
284724799198,10cyclictest2926066-21irqcore20:25:160
284724799196,5cyclictest2860079-21kthreadcore19:20:170
2847247991916,2cyclictest2868083-21kworker/u8:0+flush-259:019:35:210
2847247991914,4cyclictest3024639-21/usr/sbin/munin22:00:210
2847247991911,5cyclictest2863261-21perl19:25:140
2847257991816,1cyclictest0-21swapper/322:44:593
2847257991814,3cyclictest2930735-21kworker/u8:3+flush-259:021:05:173
2847257991813,4cyclictest3045385-21/usr/sbin/munin22:20:153
2847253991815,2cyclictest2883694-21kworker/u8:1+events_unbound20:25:182
2847253991813,4cyclictest3071633-21kworker/u8:0+flush-259:022:45:192
2847253991813,3cyclictest3008870-21cron21:45:002
2847253991811,5cyclictest2977888-21cp21:15:022
284725199184,4cyclictest3006580-21grep21:40:181
284725199183,4cyclictest3047032-21cut22:20:181
2847251991816,1cyclictest2993565-21kworker/u8:5+events_unbound21:55:191
2847251991812,4cyclictest3186836-21latency_hist00:35:011
284724799187,10cyclictest0-21swapper/023:45:190
284724799187,10cyclictest0-21swapper/021:40:010
284724799186,4cyclictest3179772-21tr00:25:180
284724799186,3cyclictest2988445-21apt-get21:25:120
2847247991814,3cyclictest3161077-21cat00:10:010
2847247991810,6cyclictest3097502-21latency_hist23:10:000
284724799180,12cyclictest3102790-21/usr/sbin/munin23:15:000
2847257991714,2cyclictest3060601-21sh22:35:003
284725799171,15cyclictest335557-21systemd-journal22:54:593
2847257991711,4cyclictest2960967-21ntpq20:55:203
2847253991713,3cyclictest3119836-21kworker/u8:3+flush-259:000:00:172
2847253991713,3cyclictest2814953-21kworker/u8:2+flush-259:019:15:002
2847253991713,2cyclictest3129974-21kworker/u8:2+events_unbound23:55:172
284725199174,4cyclictest2933681-21cut20:30:171
284725199173,3cyclictest3003757-21systemctl21:40:001
2847251991713,3cyclictest509-21rs:main1
2847251991712,4cyclictest335557-21systemd-journal00:26:061
2847251991712,3cyclictest3176493-21latency_hist00:25:001
2847251991712,3cyclictest3119742-21gpgv23:30:001
2847251991711,4cyclictest2951553-21latency_hist20:50:001
284724799179,6cyclictest3050725-21cpuspeed_turbos22:25:130
2847247991712,4cyclictest3170479-21ntpq00:15:240
2847247991711,5cyclictest2920181-21tr20:20:140
2847247991711,4cyclictest2904692-21cut20:05:130
2847247991710,6cyclictest3066759-21fw_forwarded_lo22:40:140
2847247991710,5cyclictest3060903-21latency_hist22:35:010
2847247991710,5cyclictest2909959-21fschecks_time20:10:130
2847257991614,1cyclictest2988164-21cron21:25:003
2847257991613,2cyclictest3171344-21turbostat00:20:013
2847257991612,3cyclictest335557-21systemd-journal21:39:553
2847253991613,2cyclictest3080803-21kworker/u8:3+flush-259:022:55:262
284725199163,3cyclictest2993294-21chrt21:29:451
2847251991613,2cyclictest3171317-21idleruntime-cro00:20:001
2847251991612,3cyclictest3145473-21idleruntime-cro23:55:001
2847251991611,4cyclictest335557-21systemd-journal20:02:261
2847251991611,4cyclictest3071675-21apt-get22:45:141
284724799169,6cyclictest3076814-21apt-get22:50:120
284724799169,6cyclictest2951142-21unixbench_singl20:45:250
284724799169,5cyclictest3130105-21latency_hist23:40:000
284724799169,5cyclictest3103038-21acpi23:15:140
284724799169,5cyclictest3071907-21fschecks_time22:45:170
284724799169,5cyclictest3029892-21cpu22:05:130
284724799169,5cyclictest3009059-21sed21:45:020
284724799168,6cyclictest2935133-21timerwakeupswit20:30:230
284724799166,3cyclictest2889189-21awk19:50:130
2847247991614,1cyclictest60350irq/132-enp0s29f1:rx-000:32:490
2847247991611,4cyclictest2879165-21fw_conntrack19:40:160
2847247991611,3cyclictest3144563-21ntpq23:50:210
2847247991611,3cyclictest2976512-21ntpq21:10:210
2847247991610,5cyclictest2997812-21/usr/sbin/munin21:30:230
2847247991610,5cyclictest2852934-21smartctl19:15:160
2847247991610,4cyclictest3119201-21sh23:30:010
2847247991610,4cyclictest3084904-21grep22:55:180
2847247991610,4cyclictest3049311-21needreboot22:20:210
2847247991610,4cyclictest3035304-21uniq22:10:140
2847247991610,4cyclictest2851392-21grep19:10:180
2847257991512,2cyclictest3050703-21cpuspeed_turbos22:25:133
2847253991511,3cyclictest3071633-21kworker/u8:0+flush-259:023:10:242
2847253991511,3cyclictest2929645-21kworker/u8:0+flush-259:020:50:172
284725199155,9cyclictest3119464-21cat23:30:011
2847251991513,1cyclictest0-21swapper/123:10:011
2847251991512,2cyclictest3132521-21tr23:40:191
2847251991511,3cyclictest2868120-21apt-get19:30:111
2847251991511,3cyclictest2852379-21kworker/u8:1+flush-259:019:20:001
284724799159,4cyclictest2978216-21expr21:15:140
284724799158,5cyclictest3013235-21ld21:45:250
284724799158,5cyclictest2884721-21sed19:45:170
2847247991513,1cyclictest60350irq/132-enp0s29f1:rx-000:35:190
2847247991510,4cyclictest2903662-21/usr/sbin/munin20:00:250
2847247991510,3cyclictest3161292-21apt-get00:10:120
2847247991510,3cyclictest3135110-21cron23:45:010
284725799149,4cyclictest2899171-21latency_hist20:00:013
284725799149,3cyclictest2977418-21apt-get21:15:033
2847257991412,1cyclictest3166146-21cron00:15:003
2847257991411,2cyclictest2993565-21kworker/u8:5+flush-259:021:55:253
2847257991411,2cyclictest2899538-21cpuspeed_turbos20:00:143
2847257991411,1cyclictest0-21swapper/322:55:193
2847257991410,2cyclictest2847532-21cpuspeed_turbos19:10:123
284725399149,4cyclictest335557-21systemd-journal22:15:012
2847253991411,2cyclictest2972471-21/usr/sbin/munin21:10:162
2847253991410,3cyclictest2852379-21kworker/u8:1+flush-259:019:16:102
284725199144,8cyclictest3034780-21latency_hist22:10:011
284725199141,3cyclictest0-21swapper/123:10:211
2847251991411,2cyclictest3013626-21unixbench_singl21:45:281
284725799139,3cyclictest2967289-21kworker/u8:2+flush-259:021:28:103
284725799131,3cyclictest3150639-21awk00:00:003
2847257991311,1cyclictest3140298-21cron23:50:013
2847257991310,2cyclictest3040663-21sed22:15:183
2847257991310,2cyclictest2868203-21cpuspeed_turbos19:30:133
284725399139,3cyclictest3071633-21kworker/u8:0+flush-259:023:23:372
284725399139,3cyclictest3071633-21kworker/u8:0+flush-259:023:23:362
284725399138,4cyclictest3080863-21ntpq22:50:212
2847253991311,1cyclictest3161032-21idleruntime-cro00:10:002
2847253991310,2cyclictest3130286-21cpuspeed_turbos23:40:132
2847253991310,2cyclictest3096735-21unixbench_multi23:05:252
2847253991310,2cyclictest2967447-21cpuspeed_turbos21:05:142
2847253991310,2cyclictest2962563-21irqcore21:00:152
2847253991310,2cyclictest2837147-21kworker/u8:4+flush-259:019:44:092
2847253991310,2cyclictest0-21swapper/222:20:002
284725399130,12cyclictest1-21systemd21:35:012
284725199139,3cyclictest2920090-21apt-get20:20:131
284725199138,3cyclictest2862740-21turbostat.cron19:25:001
284725199137,4cyclictest3166159-21idleruntime-cro00:15:011
284725199137,4cyclictest3149827-21ntpq23:55:231
284725199136,5cyclictest2867846-21basename19:30:001
284725199134,6cyclictest3040123-21latency_hist22:15:021
284725199134,6cyclictest2935979-21chrt20:35:121
284725199133,9cyclictest0-21swapper/122:00:181
284725199132,3cyclictest3154879-21ntpq00:00:201
2847251991310,2cyclictest3129475-21unixbench_multi23:35:241
2847251991310,2cyclictest3107495-21unixbench_multi23:15:261
284725799129,1cyclictest0-21swapper/322:10:123
284725799128,3cyclictest2993565-21kworker/u8:5+flush-259:022:07:103
284725799121,3cyclictest3076554-21latency_hist22:50:003
284725399129,2cyclictest3140765-21fschecks_count23:50:142
284725399129,2cyclictest3128943-21needreboot23:35:192
284725399129,2cyclictest2930735-21kworker/u8:3+flush-259:020:56:352
284725399128,3cyclictest2998774-21kworker/u8:1+flush-259:021:55:002
284725399128,3cyclictest2883694-21kworker/u8:1+flush-259:020:13:162
284725399127,3cyclictest335557-21systemd-journal19:30:002
2847253991210,1cyclictest0-21swapper/220:25:002
284725199128,3cyclictest2978085-21/usr/sbin/munin21:15:161
284725199127,3cyclictest3139378-21ntpq23:45:221
284725199127,3cyclictest3019201-21cron21:55:001
284725199127,3cyclictest2909476-21apt20:10:001
284725199126,4cyclictest335557-21systemd-journal20:15:011
284725199125,5cyclictest3108936-21cut23:20:141
284725199125,5cyclictest3108936-21cut23:20:141
284725199125,5cyclictest2993877-21tune2fs21:30:141
284725199125,5cyclictest2961415-21unixbench_multi20:55:241
284725199125,5cyclictest2884393-21irqcore19:45:171
284725199121,2cyclictest0-21swapper/100:40:011
2847251991210,1cyclictest3161007-21cron00:10:001
284725799119,1cyclictest0-21swapper/323:50:133
284725799119,1cyclictest0-21swapper/322:35:163
284725799118,1cyclictest0-21swapper/320:25:183
284725799117,3cyclictest3165842-21kworker/u8:0+flush-259:000:36:363
284725799117,2cyclictest0-21swapper/321:40:243
284725799116,3cyclictest2887871-21grep19:45:203
284725799111,8cyclictest2919829-21latency_hist20:20:013
284725799110,9cyclictest3029605-21cron22:05:013
284725399118,2cyclictest2904987-21irqcore20:05:162
284725399117,2cyclictest2873020-21latency_hist19:35:012
284725199118,2cyclictest2915027-21cpuspeed_turbos20:15:151
284725199115,4cyclictest2967441-21cpuspeed_turbos21:05:141
284725199115,4cyclictest2951906-21cpuspeed_turbos20:50:131
284725199115,4cyclictest2847579-21grep19:10:111
284725199114,5cyclictest2889631-21cut19:50:171
284725199114,5cyclictest2879123-21tune2fs19:40:151
284725799106,3cyclictest2883694-21kworker/u8:1+flush-259:020:10:143
284725799103,5cyclictest335557-21systemd-journal00:10:003
284725399108,1cyclictest2858707-21irqcore19:20:162
284725399108,1cyclictest0-21swapper/222:40:252
284725399108,1cyclictest0-21swapper/221:25:002
284725399108,1cyclictest0-21swapper/220:15:182
284725399108,1cyclictest0-21swapper/200:10:152
284725399107,2cyclictest0-21swapper/220:40:002
284725399107,2cyclictest0-21swapper/200:35:172
28472579996,2cyclictest2852714-21cpuspeed_turbos19:15:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional