You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 14:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sat Jul 25, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3845526995025,21cyclictest0-21swapper/100:30:151
384552699501,31cyclictest3894303-21cpu21:30:151
384552699481,41cyclictest3918369-21apt-get22:40:141
3845530994716,10cyclictest3956610-21awk00:30:122
3845533994633,5cyclictest3941270-21kworker/u8:1+events_unbound23:53:053
3845530994615,9cyclictest3899792-21fschecks_time21:45:162
384552699461,27cyclictest3951846-21memory00:15:171
3845533994513,7cyclictest131rcu_preempt23:15:123
384552699452,38cyclictest3916810-21fschecks_count22:35:131
384552699451,40cyclictest3854579-21apt-get19:35:061
384552699451,39cyclictest3935632-21apt-get23:30:141
384552699451,39cyclictest3909822-21apt-get22:15:131
384552699451,25cyclictest3939389-21irqstats23:40:181
384552699450,40cyclictest3949702-21apt-get00:10:131
3845522994536,6cyclictest3915933-21chrt22:30:260
384552699442,37cyclictest3953124-21apt-get00:20:131
384552699442,37cyclictest3851413-21apt-get19:25:141
384552699441,39cyclictest53-21kcompactd020:54:021
3845533994334,6cyclictest208-21systemd-logind20:41:523
3845530994336,4cyclictest3928583-21munin-plugin-st23:10:012
3845530994329,11cyclictest3874319-21/usr/sbin/munin20:30:202
3845530994329,10cyclictest311ktimers/200:15:112
384552699431,39cyclictest3941401-21/usr/sbin/munin23:45:181
384552699431,39cyclictest3929243-21/usr/sbin/munin23:10:181
384552699431,37cyclictest3906422-21apt-get22:05:111
384552699431,37cyclictest3863561-21apt-get20:00:121
384552699431,37cyclictest3863561-21apt-get20:00:111
384552699431,34cyclictest3946834-21grep00:00:211
384552699431,27cyclictest3867074-21df_inode20:10:141
384552699431,21cyclictest3901405-21df21:50:151
384552699430,38cyclictest3872087-21apt-get20:25:101
3845530994216,10cyclictest3850413-21mailstats19:20:222
384552699423,34cyclictest3896084-21diskstats21:35:151
384552699422,38cyclictest3923686-21apt-get22:55:121
384552699421,38cyclictest3926472-21/usr/sbin/munin23:00:241
384552699421,38cyclictest3878049-21/usr/sbin/munin20:40:241
384552699421,38cyclictest3876786-21cron20:38:001
384552699421,37cyclictest3945865-21awk00:00:001
384552699421,37cyclictest3932838-21/usr/sbin/munin23:20:221
384552699421,37cyclictest3861962-21/usr/sbin/munin19:55:161
384552699421,37cyclictest3856797-21/usr/sbin/munin19:40:111
384552699421,36cyclictest3927106-21apt-get23:05:131
384552699421,36cyclictest3889657-21/usr/sbin/munin21:15:191
384552699421,36cyclictest3868668-21apt-get20:15:151
384552699421,36cyclictest3860673-21/usr/sbin/munin19:50:211
384552699421,36cyclictest3860673-21/usr/sbin/munin19:50:211
3845522994231,7cyclictest0-21swapper/022:44:590
3845530994124,13cyclictest311ktimers/220:35:142
384552699412,33cyclictest3933928-21/usr/sbin/munin23:25:151
384552699411,37cyclictest3943951-21cron23:53:001
384552699411,37cyclictest3921539-21apt-get22:50:051
384552699411,37cyclictest3920908-21/usr/sbin/munin22:45:241
384552699411,37cyclictest3920908-21/usr/sbin/munin22:45:241
384552699411,37cyclictest3886572-21/usr/sbin/munin21:05:241
384552699411,37cyclictest3848591-21/usr/sbin/munin19:15:221
384552699411,35cyclictest3954827-21apt-get00:25:141
384552699411,35cyclictest3954827-21apt-get00:25:141
384552699411,35cyclictest3902999-21apt-get21:55:131
384552699411,35cyclictest3870721-21irqstats20:20:171
384552699411,35cyclictest3870721-21irqstats20:20:161
384552699411,35cyclictest3858445-21apt-get19:45:121
384552699411,34cyclictest3937328-21apt-get23:35:111
3845533994033,5cyclictest3874979-21kworker/u8:0+events_unbound20:48:423
3845533994031,7cyclictest391ktimers/323:30:133
3845530994028,8cyclictest3851413-21apt-get19:25:152
3845530994021,14cyclictest311ktimers/219:50:132
3845530994021,14cyclictest311ktimers/219:50:132
384552699401,37cyclictest53-21kcompactd022:27:261
384552699401,36cyclictest3959026-21/usr/sbin/munin00:35:251
384552699401,36cyclictest3909415-21cron22:13:001
384552699401,36cyclictest3905392-21/usr/sbin/munin22:00:231
384552699401,35cyclictest3915233-21/usr/sbin/munin22:30:161
384552699401,35cyclictest3911883-21irqstats22:20:181
384552699401,35cyclictest3892655-21/usr/sbin/munin21:25:151
384552699401,35cyclictest3883009-21/usr/sbin/munin20:55:241
384552699401,34cyclictest3930509-21apt-get23:15:131
384552699401,34cyclictest3878911-21apt-get20:45:121
384552699401,33cyclictest3873790-21apt-get20:30:131
384552699400,38cyclictest3849709-21apt-get19:20:111
384552699400,35cyclictest3887441-21apt-get21:10:141
3845533993932,4cyclictest3852889-21munin-plugin-st19:30:003
3845530993931,5cyclictest3884238-21fschecks_time21:00:182
3845530993925,11cyclictest3959069-21/usr/sbin/munin00:35:252
3845530993925,10cyclictest3893237-21/usr/sbin/munin21:25:242
384552699392,34cyclictest3891494-21/usr/sbin/munin21:20:221
384552699392,32cyclictest3899598-21apt-get21:45:121
3845526993914,22cyclictest3884251-21fschecks_time21:00:191
384552699391,35cyclictest3897433-21apt-get21:40:021
384552699391,35cyclictest3853219-21df19:30:151
384552699391,32cyclictest3865265-21apt-get20:05:101
3845522993916,19cyclictest0-21swapper/000:00:010
3845530993829,4cyclictest0-21swapper/220:50:162
3845530993825,10cyclictest3950074-21/usr/sbin/munin00:10:182
3845530993823,11cyclictest3878507-21cron20:43:002
3845530993823,10cyclictest3937088-21cron23:35:002
3845530993822,11cyclictest3846291-21/usr/sbin/munin19:10:222
3845530993821,9cyclictest3946835-21tr00:00:212
3845530993821,11cyclictest3905264-21tr22:00:192
3845530993818,14cyclictest3923847-21fschecks_count22:55:162
3845522993828,6cyclictest0-21swapper/021:45:260
3845533993730,4cyclictest0-21swapper/321:45:013
3845533993729,6cyclictest3956049-21kworker/u8:1+events_unbound00:28:543
3845533993729,6cyclictest3956049-21kworker/u8:1+events_unbound00:28:543
3845533993715,4cyclictest131rcu_preempt23:25:253
3845530993729,5cyclictest3920069-21apt-get22:45:132
3845530993729,5cyclictest3920069-21apt-get22:45:122
3845530993729,4cyclictest0-21swapper/220:20:012
3845530993728,5cyclictest0-21swapper/223:45:002
3845530993727,7cyclictest3901868-21/usr/sbin/munin21:50:222
3845530993725,9cyclictest3942659-21/usr/sbin/munin23:50:252
3845530993724,10cyclictest3883730-21sshd20:59:292
3845530993722,12cyclictest3853112-21apt-get19:30:112
3845530993715,15cyclictest3946116-21ls00:00:012
384552699371,33cyclictest3846274-21/usr/sbin/munin19:10:231
384552299379,9cyclictest3860831-21find19:50:220
384552299379,9cyclictest3860831-21find19:50:220
3845522993728,6cyclictest3905724-21unixbench_multi22:00:250
3845522993727,6cyclictest0-21swapper/023:10:240
3845522993723,12cyclictest121ktimers/023:20:120
3845522993712,9cyclictest3883665-21taskset20:58:390
3845522993710,9cyclictest3867709-21grep20:10:230
384553399360,5cyclictest0-21swapper/321:45:123
3845530993627,7cyclictest3857193-21/usr/sbin/munin19:40:182
3845530993626,7cyclictest3953754-21/usr/sbin/munin00:20:242
3845530993626,7cyclictest3903445-21/usr/sbin/munin21:55:202
3845530993625,8cyclictest3955420-21/usr/sbin/munin00:25:242
3845530993625,8cyclictest3955420-21/usr/sbin/munin00:25:242
3845530993625,8cyclictest3937958-21/usr/sbin/munin23:35:212
3845530993623,9cyclictest3865711-21/usr/sbin/munin20:05:192
3845530993623,10cyclictest3912151-21/usr/sbin/munin22:20:232
3845530993623,10cyclictest3887113-21cron21:09:002
3845530993622,11cyclictest3910056-21/usr/sbin/munin22:15:162
3845530993622,10cyclictest3907882-21cron22:10:012
3845530993620,12cyclictest438-21cron23:15:012
3845530993619,12cyclictest3941993-21grep23:45:242
3845530993619,12cyclictest3918117-21apt22:40:002
384552299369,9cyclictest3952873-21latency_hist00:20:010
3845533993528,4cyclictest0-21swapper/322:15:243
3845533993527,6cyclictest3856735-21kworker/u8:1+events_unbound19:51:453
3845533993527,6cyclictest3856735-21kworker/u8:1+events_unbound19:51:453
3845533993524,6cyclictest0-21swapper/320:00:263
3845533993524,6cyclictest0-21swapper/320:00:263
384553099355,21cyclictest0-21swapper/220:11:132
3845530993526,7cyclictest3890954-21/usr/sbin/munin21:20:142
3845530993526,6cyclictest3888904-21sh21:15:002
3845530993524,8cyclictest3855866-21/usr/sbin/munin19:35:232
3845530993524,7cyclictest3863567-21/usr/sbin/munin20:00:132
3845530993524,7cyclictest3863567-21/usr/sbin/munin20:00:122
3845530993523,7cyclictest3913358-21/usr/sbin/munin22:25:132
3845530993522,8cyclictest3925401-21/usr/sbin/munin23:00:182
3845530993521,9cyclictest3919829-21cron22:45:002
3845530993521,10cyclictest3849054-21head19:15:262
3845530993520,9cyclictest3909580-21cron22:15:012
3845530993520,12cyclictest3896568-21/usr/sbin/munin21:35:212
3845530993520,11cyclictest3933261-21grep23:20:242
3845530993520,11cyclictest3871846-21cron20:25:012
3845530993520,11cyclictest3871846-21cron20:25:002
3845530993519,9cyclictest3862057-21fschecks_time19:55:162
3845530993519,11cyclictest3873073-21tail20:25:242
3845522993522,10cyclictest121ktimers/019:25:130
3845522993511,5cyclictest11-21ksoftirqd/019:45:250
3845533993428,4cyclictest3935292-21kworker/u8:0+events_unbound00:06:393
3845533993428,4cyclictest3895660-21kworker/u8:3+events_unbound21:57:013
3845533993427,5cyclictest3907774-21kworker/u8:1+events_unbound22:27:103
3845530993426,4cyclictest0-21swapper/221:40:242
3845530993421,8cyclictest3915998-21grep22:30:252
3845530993421,10cyclictest3931257-21/usr/sbin/munin23:15:232
3845530993420,9cyclictest3948114-21/usr/sbin/munin00:05:162
3845530993419,9cyclictest3923450-21apt22:55:002
3845530993419,9cyclictest3890190-21head21:15:252
3845530993419,12cyclictest3894939-21/usr/sbin/munin21:30:252
3845522993426,5cyclictest0-21swapper/020:30:000
3845522993425,6cyclictest0-21swapper/023:39:590
384553399339,13cyclictest131rcu_preempt19:30:253
3845533993327,3cyclictest3876980-21sendmail20:39:593
3845533993325,6cyclictest3885294-21kworker/u8:2+events_unbound21:06:563
3845533993323,7cyclictest3897439-21sendmail21:40:013
3845533993316,6cyclictest3895660-21kworker/u8:3+events_unbound21:53:463
3845530993321,9cyclictest3934770-21/usr/sbin/munin23:25:242
3845530993319,9cyclictest3859463-21unixbench_multi19:45:252
3845530993317,12cyclictest3879106-21grep20:45:142
3845522993322,6cyclictest0-21swapper/023:15:250
3845522993321,8cyclictest3926422-21grep23:00:250
3845533993223,7cyclictest3885294-21kworker/u8:2+events_unbound21:04:413
3845533993223,5cyclictest0-21swapper/322:55:263
3845533993219,5cyclictest131rcu_preempt21:00:013
3845533993216,10cyclictest391ktimers/319:20:113
3845522993222,7cyclictest3857745-21grep19:40:240
384553399318,6cyclictest131rcu_preempt00:35:273
3845533993124,4cyclictest0-21swapper/319:40:153
3845533993117,6cyclictest3952535-21kworker/u8:3+events_unbound00:34:393
3845533993110,7cyclictest131rcu_preempt20:05:243
3845522993124,4cyclictest0-21swapper/000:10:000
3845522993123,5cyclictest3924695-21grep22:55:260
3845522993121,7cyclictest3949838-21grep00:10:170
3845522993119,9cyclictest121ktimers/022:50:130
3845522993119,8cyclictest3917656-21grep22:35:230
3845522993118,10cyclictest121ktimers/022:45:130
3845522993118,10cyclictest121ktimers/022:45:120
3845522993117,10cyclictest3909977-21grep22:15:150
3845522993111,7cyclictest3942561-21date23:50:010
384553399308,5cyclictest3921112-21grep22:45:243
384553399308,5cyclictest3921112-21grep22:45:243
3845533993019,8cyclictest391ktimers/319:45:133
3845533993010,12cyclictest131rcu_preempt22:00:243
384552299309,17cyclictest3856635-21cat19:40:010
384552299306,19cyclictest3899508-21cat21:45:020
384552299303,9cyclictest0-21swapper/022:25:240
3845522993025,2cyclictest0-21swapper/021:25:120
3845522993024,3cyclictest0-21swapper/000:30:250
3845522993020,7cyclictest3940769-21apt-config23:45:010
3845522993017,9cyclictest3845993-21if_eth119:10:180
3845522993010,12cyclictest0-21swapper/000:20:170
384553399296,7cyclictest40-21ksoftirqd/319:40:013
3845533992923,3cyclictest0-21swapper/300:20:003
3845533992922,4cyclictest391ktimers/321:20:253
3845533992921,4cyclictest0-21swapper/300:10:263
3845533992917,4cyclictest131rcu_preempt23:00:243
384552299298,6cyclictest3871869-21idleruntime-cro20:25:010
384552299298,6cyclictest3871869-21idleruntime-cro20:25:010
3845522992922,4cyclictest3890865-21apt-get21:20:120
3845522992922,4cyclictest0-21swapper/022:15:000
3845522992912,13cyclictest3863318-21idleruntime-cro20:00:010
384553399288,12cyclictest0-21swapper/322:35:193
384553399286,3cyclictest131rcu_preempt23:10:123
3845533992822,4cyclictest3907774-21kworker/u8:1+events_unbound22:09:423
3845533992820,6cyclictest3907774-21kworker/u8:1+events_unbound22:10:413
3845533992820,4cyclictest0-21swapper/320:00:013
384553399280,8cyclictest3874846-21grep20:30:253
3845522992819,6cyclictest0-21swapper/023:25:250
3845522992817,8cyclictest3864590-21/usr/sbin/munin20:00:260
3845522992817,8cyclictest3864590-21/usr/sbin/munin20:00:260
3845522992816,9cyclictest3853346-21/usr/sbin/munin19:30:170
3845522992815,9cyclictest3904030-21grep21:55:270
3845522992815,7cyclictest3885037-21grep21:00:260
3845522992815,10cyclictest3958789-21/usr/sbin/munin00:35:220
3845522992815,10cyclictest3850469-21/usr/sbin/munin19:20:220
3845522992811,13cyclictest3889812-21ps21:15:230
3845522992810,13cyclictest3886772-21grep21:05:250
3845533992721,4cyclictest3945712-21kworker/u8:2+events_unbound00:03:133
3845533992720,5cyclictest3890280-21kworker/u8:1+events_unbound21:25:583
3845533992720,4cyclictest3919849-21munin-plugin-st22:45:003
3845533992720,4cyclictest0-21swapper/321:30:273
3845533992720,3cyclictest0-21swapper/319:20:013
3845533992719,4cyclictest0-21swapper/300:00:013
3845533992718,7cyclictest3874979-21kworker/u8:0+events_unbound20:50:043
384552299274,20cyclictest3877462-21basename20:40:160
384552299273,9cyclictest0-21swapper/020:35:250
3845522992716,7cyclictest0-21swapper/019:20:010
3845522992715,8cyclictest3928566-21cron23:10:000
3845522992714,9cyclictest3882077-21sh20:55:010
3845522992714,10cyclictest3888201-21/usr/sbin/munin21:10:250
3845522992713,11cyclictest3894567-21/usr/sbin/munin21:30:190
3845522992712,10cyclictest204-21dbus-daemon22:07:030
3845522992711,12cyclictest3936648-21grep23:30:250
3845522992711,11cyclictest3943287-21grep23:50:220
3845522992711,11cyclictest3902346-21grep21:50:250
3845522992711,10cyclictest3946833-21ntpq00:00:200
3845522992710,13cyclictest3897434-21latency_hist21:40:010
384553399262,12cyclictest0-21swapper/322:32:503
384553399261,5cyclictest3940054-21tail23:40:263
384553399260,13cyclictest3938797-21cron23:39:593
3845522992620,3cyclictest3954827-21apt-get00:25:130
3845522992620,3cyclictest3954827-21apt-get00:25:120
3845522992616,7cyclictest3912125-21/usr/sbin/munin22:20:220
3845522992615,8cyclictest3879236-21/usr/sbin/munin20:45:170
3845522992615,8cyclictest3865936-21/usr/sbin/munin20:05:220
3845522992613,9cyclictest204-21dbus-daemon20:16:480
3845522992611,8cyclictest3874785-21grep20:30:240
384553399254,5cyclictest3867967-21grep20:10:243
3845533992519,4cyclictest3890280-21kworker/u8:1+events_unbound21:16:553
3845533992518,4cyclictest0-21swapper/320:15:263
3845533992517,5cyclictest3846889-21kworker/u8:0+events_unbound19:13:093
3845533992517,4cyclictest0-21swapper/323:05:153
384553399251,6cyclictest40-21ksoftirqd/322:50:263
3845533992516,6cyclictest391ktimers/322:25:003
384553399251,15cyclictest0-21swapper/320:20:133
384553399251,15cyclictest0-21swapper/320:20:133
384553399251,13cyclictest0-21swapper/300:20:413
384553399251,12cyclictest0-21swapper/323:20:593
3845533992510,5cyclictest131rcu_preempt21:10:263
384553399247,5cyclictest131rcu_preempt20:25:213
384553399238,3cyclictest0-21swapper/323:45:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional