You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 07:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Thu Apr 23, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3594548996051,5cyclictest3701919-21munin-plugin-st00:20:000
3594561995244,5cyclictest1-21systemd21:56:053
3594561995221,12cyclictest3659426-21ntp_states22:15:213
3594557995244,4cyclictest3629416-21idleruntime-cro20:50:002
3594552994943,4cyclictest3693027-21kworker/u8:3+events_unbound00:22:281
3594548994841,4cyclictest3634065-21grep21:00:270
3594561994617,10cyclictest3626355-21df20:40:143
3594561994617,10cyclictest3610994-21df_inode19:55:173
3594552994640,4cyclictest3653493-21kworker/u8:3+events_unbound22:36:131
3594548994640,4cyclictest3675463-21kworker/u8:3+events_unbound23:11:230
3594561994525,12cyclictest3703848-21cron00:24:593
3594561994519,7cyclictest3600557-21df_inode19:25:153
3594561994435,5cyclictest3638179-21apt-get21:15:133
3594561994418,9cyclictest3643069-21apt-get21:30:053
3594561994418,9cyclictest3643069-21apt-get21:30:043
3594557994436,5cyclictest311ktimers/222:00:122
3594552994437,5cyclictest3653493-21kworker/u8:3+events_unbound22:21:081
3594552994437,4cyclictest3701920-21grep00:20:001
3594561994333,7cyclictest391ktimers/321:25:123
3594561994332,8cyclictest3685068-21/usr/sbin/munin23:30:233
3594561994332,7cyclictest3655462-21apt-get22:05:123
3594561994330,8cyclictest3628288-21/usr/sbin/munin20:45:183
3594561994329,10cyclictest3594711-21apt-get19:10:103
3594561994329,10cyclictest3594711-21apt-get19:10:103
3594561994325,12cyclictest3681195-21cut23:20:163
3594557994332,7cyclictest3615364-21grep20:05:252
3594548994334,6cyclictest3703375-21unixbench_multi00:20:250
3594561994236,3cyclictest3657161-21apt-get22:10:123
3594561994236,3cyclictest3648637-21apt-get21:45:113
3594561994232,6cyclictest3636476-21apt-get21:10:143
3594561994231,8cyclictest3665688-21apt-get22:35:143
3594561994228,11cyclictest3629772-21/usr/sbin/munin20:50:143
3594552994236,4cyclictest3668729-21kworker/u8:3+events_unbound22:56:161
3594552994236,4cyclictest3668729-21kworker/u8:3+events_unbound22:56:161
3594552994236,4cyclictest3653493-21kworker/u8:3+events_unbound22:26:101
3594552994228,10cyclictest231ktimers/120:00:131
3594548994217,22cyclictest121ktimers/019:50:150
3594561994135,3cyclictest3624529-21apt-get20:35:123
3594561994135,3cyclictest3624529-21apt-get20:35:123
3594561994134,4cyclictest3646937-21apt-get21:40:123
3594561994134,4cyclictest3609179-21apt-get19:50:143
3594561994129,9cyclictest3705792-21apt-get00:30:123
3594561994127,9cyclictest3650942-21/usr/sbin/munin21:50:223
3594548994114,20cyclictest131rcu_preempt21:55:190
3594561994034,3cyclictest3605778-21apt-get19:40:123
3594561994033,3cyclictest3697037-21apt-get00:05:133
3594561994033,3cyclictest3677632-21apt-get23:10:133
3594561994033,3cyclictest3670811-21apt-get22:50:133
3594561994030,7cyclictest3687446-21/usr/sbin/munin23:35:263
3594561994030,7cyclictest3633191-21/usr/sbin/munin21:00:173
3594561994029,8cyclictest3679781-21/usr/sbin/munin23:15:183
3594561994028,9cyclictest3669625-21/usr/sbin/munin22:45:193
3594561994028,8cyclictest3645214-21apt-get21:35:123
3594561994028,7cyclictest3612720-21diskstats20:00:163
3594561994027,10cyclictest3664614-21/usr/sbin/munin22:30:233
3594561994027,10cyclictest3661803-21sshd22:22:043
3594561994025,11cyclictest3632419-21/usr/sbin/munin20:55:273
359456199401,36cyclictest3636306-21latency_hist21:10:013
3594561993932,4cyclictest3675935-21apt-get23:05:133
3594561993932,4cyclictest3621125-21apt-get20:25:133
3594561993932,3cyclictest3614300-21apt-get20:05:123
3594561993928,8cyclictest3690562-21/usr/sbin/munin23:45:203
3594561993928,8cyclictest3641114-21cron21:22:003
3594561993928,8cyclictest3597557-21/usr/sbin/munin19:15:203
3594561993928,7cyclictest3693728-21diskstats23:55:143
3594561993928,7cyclictest3674219-21sshd23:00:043
3594561993927,9cyclictest3607818-21/usr/sbin/munin19:45:183
3594561993927,9cyclictest3598793-21/usr/sbin/munin19:20:133
3594561993926,9cyclictest3616114-21/usr/sbin/munin20:10:143
3594561993926,8cyclictest3623168-21/usr/sbin/munin20:30:193
3594561993925,10cyclictest3673091-21/usr/sbin/munin22:55:223
3594561993925,10cyclictest3673091-21/usr/sbin/munin22:55:223
3594561993924,11cyclictest3603981-21grep19:35:153
3594561993923,12cyclictest3708102-21/usr/sbin/munin00:35:233
3594561993923,11cyclictest3602327-21fschecks_time19:30:173
3594548993917,15cyclictest131rcu_preempt21:20:200
3594561993832,3cyclictest3617701-21apt-get20:15:133
3594561993830,5cyclictest3704092-21apt-get00:25:133
3594561993826,9cyclictest3692093-21/usr/sbin/munin23:50:163
3594561993822,12cyclictest3668467-21grep22:40:273
3594557993832,4cyclictest3680808-21cron23:20:002
3594557993810,9cyclictest3667421-21grep22:40:152
3594557993810,10cyclictest3686382-21egrep23:35:132
3594548993829,5cyclictest3698498-21auditctl00:10:010
3594548993828,6cyclictest0-21swapper/022:45:250
3594561993731,3cyclictest3682759-21apt-get23:25:113
3594561993727,7cyclictest53-21kcompactd022:01:003
3594561993727,7cyclictest3688631-21/usr/sbin/munin23:40:223
3594561993723,10cyclictest3620463-21head20:20:263
3594561993721,11cyclictest3696347-21grep00:00:283
3594561993720,14cyclictest3663753-21munin-plugin-st22:30:013
3594557993729,4cyclictest0-21swapper/220:40:252
3594552993730,5cyclictest3693027-21kworker/u8:3+events_unbound00:28:521
3594552993730,4cyclictest3659914-21unixbench_singl22:15:261
3594548993726,7cyclictest0-21swapper/021:10:280
3594548993726,6cyclictest0-21swapper/023:45:240
3594548993725,7cyclictest0-21swapper/019:55:260
3594548993724,9cyclictest0-21swapper/000:35:260
3594548993720,3cyclictest131rcu_preempt22:05:180
3594548993712,17cyclictest3606722-21timerwakeupswit19:40:270
3594561993630,3cyclictest3698740-21apt-get00:10:143
3594561993623,10cyclictest3700537-21/usr/sbin/munin00:15:133
3594557993621,12cyclictest3599260-21/usr/sbin/munin19:20:192
3594557993618,14cyclictest3660361-21idleruntime-cro22:20:002
3594557993610,9cyclictest3624632-21df20:35:142
3594557993610,9cyclictest3624632-21df20:35:142
3594552993628,4cyclictest0-21swapper/100:35:151
3594552993626,7cyclictest0-21swapper/120:55:261
3594552993620,5cyclictest131rcu_preempt00:05:161
3594552993619,5cyclictest131rcu_preempt21:20:221
3594552993618,10cyclictest131rcu_preempt21:35:161
3594552993617,5cyclictest131rcu_preempt23:55:191
3594552993616,5cyclictest131rcu_preempt21:30:231
3594552993616,5cyclictest131rcu_preempt21:30:221
3594548993620,5cyclictest0-21swapper/023:20:160
359455799358,11cyclictest3690728-21run-parts23:45:242
3594557993524,8cyclictest3704722-21ps00:25:232
3594552993523,5cyclictest131rcu_preempt21:02:001
3594552993521,5cyclictest131rcu_preempt23:50:051
3594552993519,5cyclictest3615448-21kworker/u8:0+events_unbound20:08:121
3594552993519,5cyclictest131rcu_preempt23:10:201
3594552993519,5cyclictest131rcu_preempt21:55:191
3594552993519,4cyclictest131rcu_preempt00:00:251
3594552993518,9cyclictest131rcu_preempt19:31:491
3594552993516,9cyclictest131rcu_preempt20:15:241
3594552993515,10cyclictest131rcu_preempt22:05:191
3594552993513,4cyclictest131rcu_preempt23:00:231
3594548993525,6cyclictest0-21swapper/000:00:300
3594548993524,7cyclictest0-21swapper/023:25:250
3594548993524,7cyclictest0-21swapper/020:50:240
3594548993519,5cyclictest131rcu_preempt20:15:010
3594548993518,4cyclictest0-21swapper/023:30:240
3594548993517,4cyclictest0-21swapper/020:15:170
3594548993515,3cyclictest0-21swapper/021:50:160
3594557993425,4cyclictest3636258-21sh21:10:012
3594557993424,7cyclictest3702013-21ls00:20:012
3594557993420,11cyclictest3595341-21/usr/sbin/munin19:10:222
3594557993420,11cyclictest3595341-21/usr/sbin/munin19:10:212
3594557993414,11cyclictest3632808-21cron21:00:022
359455299345,15cyclictest24-21ksoftirqd/123:35:021
3594552993428,4cyclictest3670448-21kworker/u8:0+events_unbound23:06:171
3594552993428,4cyclictest3605183-21kworker/u8:2+events_unbound19:48:421
3594552993424,6cyclictest0-21swapper/122:30:261
3594552993422,4cyclictest3685845-21kworker/u8:3+events_unbound23:35:071
3594552993421,5cyclictest131rcu_preempt19:10:131
3594552993421,5cyclictest131rcu_preempt19:10:121
3594552993418,5cyclictest131rcu_preempt00:10:251
3594552993415,10cyclictest131rcu_preempt21:25:141
3594552993414,4cyclictest0-21swapper/119:16:541
3594552993414,12cyclictest131rcu_preempt23:30:021
3594548993424,6cyclictest3628947-21unixbench_multi20:45:250
3594548993423,7cyclictest0-21swapper/020:00:250
3594548993422,9cyclictest121ktimers/021:35:110
3594548993421,4cyclictest131rcu_preempt22:36:290
3594557993324,6cyclictest0-21swapper/221:15:002
3594557993319,11cyclictest3639117-21/usr/sbin/munin21:15:262
3594557993318,10cyclictest3643060-21cron21:30:002
3594557993317,12cyclictest3603199-21grep19:30:272
3594557993315,10cyclictest3703847-21cron00:24:592
359455299337,23cyclictest231ktimers/120:25:111
3594552993325,4cyclictest0-21swapper/120:20:161
3594552993322,4cyclictest131rcu_preempt21:17:461
3594552993321,6cyclictest3600070-21kworker/u8:1+events_unbound19:40:451
3594552993321,5cyclictest131rcu_preempt23:25:011
3594552993320,5cyclictest131rcu_preempt20:40:171
3594552993317,4cyclictest131rcu_preempt21:45:231
3594552993315,11cyclictest131rcu_preempt21:05:201
3594552993314,5cyclictest131rcu_preempt20:15:021
3594552993314,4cyclictest131rcu_preempt21:10:211
3594552993314,10cyclictest131rcu_preempt20:35:251
3594552993314,10cyclictest131rcu_preempt20:35:241
3594552993313,5cyclictest131rcu_preempt22:52:221
3594552993313,12cyclictest131rcu_preempt20:30:161
3594548993323,6cyclictest0-21swapper/021:25:240
3594548993322,7cyclictest0-21swapper/020:05:250
3594548993320,10cyclictest121ktimers/022:10:120
3594548993319,3cyclictest131rcu_preempt21:40:230
3594548993316,4cyclictest0-21swapper/023:05:170
3594548993314,4cyclictest131rcu_preempt22:35:010
3594548993313,5cyclictest121ktimers/022:44:210
359455799329,4cyclictest3698845-21df00:10:142
359455799327,9cyclictest3605310-21apt-get19:40:022
359455799326,9cyclictest629-21snmpd21:45:242
3594557993225,4cyclictest0-21swapper/223:55:242
3594557993222,5cyclictest0-21swapper/221:00:272
3594557993221,7cyclictest0-21swapper/222:20:242
3594557993220,9cyclictest3676909-21/usr/sbin/munin23:05:242
3594557993220,9cyclictest3652491-21/usr/sbin/munin21:55:192
3594557993217,10cyclictest3600653-21fschecks_time19:25:162
3594557993216,12cyclictest3613650-21grep20:00:292
359455299323,14cyclictest24-21ksoftirqd/123:40:251
359455299323,14cyclictest24-21ksoftirqd/120:45:151
3594552993219,7cyclictest131rcu_preempt23:15:231
3594552993217,5cyclictest131rcu_preempt23:45:201
3594552993217,4cyclictest131rcu_preempt22:50:001
3594552993216,5cyclictest131rcu_preempt22:10:241
3594552993216,5cyclictest131rcu_preempt19:35:181
3594552993215,4cyclictest131rcu_preempt22:40:281
3594552993215,10cyclictest0-21swapper/119:20:151
3594552993213,5cyclictest131rcu_preempt21:40:191
3594552993213,4cyclictest131rcu_preempt19:29:301
3594548993224,4cyclictest0-21swapper/023:52:220
3594548993218,3cyclictest131rcu_preempt20:25:240
3594548993215,4cyclictest0-21swapper/022:20:150
3594548993214,15cyclictest0-21swapper/019:30:170
3594548993212,3cyclictest0-21swapper/000:30:240
3594548993211,3cyclictest0-21swapper/020:55:180
359455799315,8cyclictest0-21swapper/222:25:162
359455799314,10cyclictest3683331-21ntpq23:25:212
3594557993121,6cyclictest0-21swapper/222:15:012
3594557993120,8cyclictest3685669-21cron23:32:002
3594557993117,8cyclictest3678618-21unixbench_multi23:10:242
3594557993117,8cyclictest3623830-21grep20:30:272
3594557993117,10cyclictest3681197-21fschecks_count23:20:172
3594557993115,13cyclictest3670094-21tail22:45:242
3594557993115,12cyclictest3673557-21head22:55:262
3594557993115,12cyclictest3673557-21head22:55:262
3594557993113,14cyclictest3604890-21head19:35:262
359455299315,13cyclictest24-21ksoftirqd/119:50:221
3594552993118,5cyclictest131rcu_preempt19:55:171
3594552993116,4cyclictest131rcu_preempt20:50:141
3594552993111,9cyclictest131rcu_preempt00:30:181
3594548993125,4cyclictest3598738-21apt19:20:120
3594548993122,7cyclictest121ktimers/021:15:260
3594548993122,6cyclictest0-21swapper/000:29:590
3594548993121,7cyclictest121ktimers/022:00:210
3594548993120,8cyclictest121ktimers/021:10:010
3594548993120,4cyclictest131rcu_preempt21:30:050
3594548993120,4cyclictest131rcu_preempt21:30:040
359454899311,8cyclictest11-21ksoftirqd/023:55:230
3594548993115,4cyclictest0-21swapper/023:00:190
3594548993110,3cyclictest0-21swapper/020:35:010
3594557993023,4cyclictest0-21swapper/223:50:172
3594557993023,3cyclictest0-21swapper/221:40:192
3594557993019,8cyclictest3689066-21/usr/sbin/munin23:40:252
3594557993019,8cyclictest3671793-21/usr/sbin/munin22:50:252
3594557993019,8cyclictest3598315-21cron19:18:002
3594557993018,10cyclictest3618841-21sshd20:16:052
3594557993017,10cyclictest3651711-21cron21:54:002
3594557993016,10cyclictest3646247-21unixbench_singl21:35:242
3594557993015,11cyclictest3644570-21head21:30:252
3594557993015,11cyclictest3644570-21head21:30:252
3594552993014,5cyclictest131rcu_preempt22:00:151
359454899308,5cyclictest0-21swapper/023:35:260
3594548993021,6cyclictest3595694-21grep19:10:230
3594548993021,6cyclictest3595694-21grep19:10:230
3594548993020,7cyclictest3659551-21seq22:15:230
3594548993019,8cyclictest121ktimers/022:51:150
3594548993014,5cyclictest131rcu_preempt21:45:130
3594548993012,4cyclictest0-21swapper/022:25:120
3594548993012,3cyclictest0-21swapper/000:10:160
359454899301,15cyclictest0-21swapper/020:35:210
359454899301,15cyclictest0-21swapper/020:35:210
359455799295,12cyclictest0-21swapper/200:05:122
3594557992921,4cyclictest0-21swapper/220:25:252
3594557992919,7cyclictest3665685-21/usr/sbin/munin22:35:252
3594557992919,7cyclictest3611601-21/usr/sbin/munin19:55:242
3594557992918,8cyclictest3619872-21memory20:20:192
3594557992918,8cyclictest3610266-21sshd19:50:352
3594557992917,9cyclictest3674357-21/usr/sbin/munin23:00:142
3594557992916,9cyclictest3696790-21cron00:05:002
3594557992916,8cyclictest3706791-21grep00:30:242
3594557992915,10cyclictest3708954-21cron00:40:002
3594557992911,13cyclictest3665596-21wc22:35:012
3594552992913,5cyclictest131rcu_preempt21:50:261
359454899299,10cyclictest0-21swapper/019:15:030
359454899297,3cyclictest0-21swapper/023:40:130
359454899296,20cyclictest121ktimers/019:25:170
3594548992919,6cyclictest0-21swapper/023:15:250
3594548992914,4cyclictest131rcu_preempt19:45:140
359454899291,11cyclictest3673035-21ntpq22:55:200
359454899291,11cyclictest3673035-21ntpq22:55:200
3594557992818,7cyclictest3639887-21sshd21:20:062
3594557992817,8cyclictest3616334-21/usr/sbin/munin20:10:172
3594557992817,7cyclictest3631125-21grep20:55:022
3594557992812,12cyclictest3608002-21tr19:45:202
3594548992823,3cyclictest151rcuc/020:42:300
3594557992716,7cyclictest3655597-21fschecks_count22:05:162
359454899271,8cyclictest11-21ksoftirqd/019:35:200
359454899270,17cyclictest0-21swapper/020:22:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional