You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 23:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Jun 12, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3866876995123,25cyclictest3963125-21proc_pri11:45:222
3866876994736,8cyclictest3926039-21basename10:00:012
3866869994536,6cyclictest231ktimers/112:35:101
3866880994416,10cyclictest3977539-21cat12:25:233
3866869994433,6cyclictest53-21kcompactd010:55:271
3866869994333,7cyclictest3920455-21grep09:40:261
386687699421,9cyclictest3957145-21cron11:30:012
386687699421,11cyclictest3884475-21users07:55:252
386687699421,10cyclictest3938176-21apt-get10:35:042
386687699421,10cyclictest3879997-21/usr/sbin/munin07:45:262
3866880994130,6cyclictest0-21swapper/308:15:253
3866880994130,6cyclictest0-21swapper/308:15:243
3866880994129,7cyclictest3897370-21/usr/sbin/munin08:35:153
3866880994128,9cyclictest3936795-21/usr/sbin/munin10:30:113
3866880994116,9cyclictest3878748-21meminfo07:40:223
386687699412,9cyclictest3893004-21/usr/sbin/munin08:20:262
3866865994128,10cyclictest121ktimers/010:40:120
3866880994033,4cyclictest3884863-21cron08:00:013
3866880994029,7cyclictest3956122-21/usr/sbin/munin11:25:173
3866880994028,8cyclictest3952267-21/usr/sbin/munin11:15:113
3866880994027,10cyclictest3938701-21/usr/sbin/munin10:35:123
386687699408,3cyclictest0-21swapper/207:50:272
386687699408,27cyclictest0-21swapper/211:35:242
386687699407,7cyclictest3944986-21seq10:52:262
386687699401,8cyclictest3887006-21cut08:05:182
386687699401,7cyclictest3946477-21grep10:55:272
3866876994017,20cyclictest0-21swapper/210:05:162
3866876994014,23cyclictest0-21swapper/209:10:252
3866869994032,5cyclictest3913653-21unixbench_multi09:20:261
3866880993929,7cyclictest3961814-21unixbench_multi11:40:263
3866880993928,7cyclictest0-21swapper/309:20:003
3866880993926,9cyclictest3916039-21/usr/sbin/munin09:30:143
3866880993925,9cyclictest3973784-21/usr/sbin/munin12:15:253
3866880993925,11cyclictest3905798-21/usr/sbin/munin09:00:283
386687699399,3cyclictest0-21swapper/208:25:172
386687699398,7cyclictest3904199-21diskstats08:55:142
386687699392,9cyclictest3966892-21unixbench_multi11:55:242
386687699392,7cyclictest3948206-21head11:00:272
386687699391,8cyclictest3936056-21unixbench_singl10:25:242
386687699391,8cyclictest3902378-21/usr/sbin/munin08:50:182
3866876993913,23cyclictest0-21swapper/211:30:152
386687699391,11cyclictest3906373-21ntp_states09:00:232
386687699391,10cyclictest3918717-21tail09:35:242
386687699391,10cyclictest3878836-21grep07:40:232
3866869993927,8cyclictest3880989-21grep07:45:281
3866865993933,4cyclictest3960552-21cron11:40:000
3866880993831,4cyclictest3957383-21apt-get11:30:103
3866880993831,3cyclictest3928185-21apt-get10:05:113
3866880993830,4cyclictest3978267-21apt-get12:30:113
3866880993830,4cyclictest0-21swapper/310:20:013
3866880993828,6cyclictest0-21swapper/308:10:283
3866880993828,6cyclictest0-21swapper/308:10:283
3866880993827,8cyclictest3965237-21/usr/sbin/munin11:50:263
3866880993827,8cyclictest3887373-21/usr/sbin/munin08:05:223
3866880993827,7cyclictest3921490-21irqstats09:45:173
3866880993826,9cyclictest3948942-21/usr/sbin/munin11:05:113
3866880993826,9cyclictest3904421-21/usr/sbin/munin08:55:183
3866880993825,10cyclictest3923295-21/usr/sbin/munin09:50:213
3866880993824,8cyclictest0-21swapper/310:00:113
3866880993823,12cyclictest3900055-21/usr/sbin/munin08:40:273
3866880993820,11cyclictest391ktimers/310:10:113
386687699387,5cyclictest0-21swapper/212:10:242
386687699387,4cyclictest3896360-21/usr/sbin/munin08:30:262
386687699386,6cyclictest3867235-21/usr/sbin/munin07:10:162
386687699382,8cyclictest3924312-21cron09:55:002
386687699382,6cyclictest3954198-21cut11:20:172
386687699382,11cyclictest3891722-21grep08:20:002
386687699382,11cyclictest3891722-21grep08:20:002
386687699382,10cyclictest3970557-21grep12:05:242
386687699381,9cyclictest3877028-21meminfo07:35:222
386687699381,8cyclictest3899976-21tail08:40:272
3866876993813,2cyclictest0-21swapper/211:40:482
386687699381,11cyclictest3968164-21/usr/sbin/munin12:00:172
386687699381,11cyclictest3927112-21ps10:00:222
3866865993828,6cyclictest3915357-21unixbench_multi09:25:260
3866880993728,6cyclictest3969521-21sshd12:05:033
3866880993727,7cyclictest3968564-21/usr/sbin/munin12:00:233
3866880993727,7cyclictest3908439-21/usr/sbin/munin09:05:263
3866880993727,7cyclictest3908439-21/usr/sbin/munin09:05:263
3866880993727,7cyclictest3892841-21/usr/sbin/munin08:20:243
3866880993727,6cyclictest3944762-21/usr/sbin/munin10:50:253
3866880993726,8cyclictest3942304-21/usr/sbin/munin10:45:173
3866880993726,8cyclictest3942304-21/usr/sbin/munin10:45:173
3866880993726,8cyclictest3919021-21cron09:38:013
3866880993726,7cyclictest3947613-21memory11:00:203
3866880993725,8cyclictest3873570-21/usr/sbin/munin07:25:273
3866880993725,8cyclictest3873570-21/usr/sbin/munin07:25:273
3866880993725,7cyclictest3871289-21/usr/sbin/munin07:20:173
3866880993724,9cyclictest0-21swapper/312:35:103
3866880993722,12cyclictest3915104-21/usr/sbin/munin09:25:253
3866880993721,12cyclictest3876914-21/usr/sbin/munin07:35:193
386687699378,3cyclictest3943027-21unixbench_multi10:45:262
386687699378,3cyclictest3943027-21unixbench_multi10:45:252
386687699371,8cyclictest3915253-21timerwakeupswit09:25:262
386687699371,7cyclictest3872116-21head07:20:262
386687699371,6cyclictest3917495-21cron09:35:012
386687699371,11cyclictest3920923-21sh09:45:012
386687699371,11cyclictest3885118-21apt-get08:00:112
386687699371,10cyclictest3937773-21sed10:30:242
386687699371,10cyclictest3921716-21ntp_states09:45:212
3866865993730,4cyclictest3919210-21munin-plugin-st09:40:000
3866865993729,4cyclictest0-21swapper/012:24:590
3866865993729,4cyclictest0-21swapper/011:20:260
3866880993629,4cyclictest3912632-21apt-get09:20:123
3866880993626,7cyclictest3946422-21/usr/sbin/munin10:55:263
3866880993626,7cyclictest3903159-21/usr/sbin/munin08:50:263
3866880993626,7cyclictest3896926-21cron08:34:013
3866880993626,7cyclictest3894845-21/usr/sbin/munin08:25:283
3866880993626,7cyclictest3870006-21/usr/sbin/munin07:15:243
3866880993625,8cyclictest3909727-21/usr/sbin/munin09:10:193
3866880993625,7cyclictest3966017-21df_inode11:55:123
3866880993625,7cyclictest3954120-21/usr/sbin/munin11:20:163
3866880993624,9cyclictest3963068-21ntp_states11:45:213
3866880993624,7cyclictest3900917-21fw_packets08:45:183
3866880993622,11cyclictest3935406-21/usr/sbin/munin10:25:173
3866880993622,11cyclictest3933752-21/usr/sbin/munin10:20:183
3866880993622,11cyclictest3882272-21/usr/sbin/munin07:50:233
3866880993621,11cyclictest3920912-21cron09:45:003
3866880993621,10cyclictest3940339-21/usr/sbin/munin10:40:113
3866880993620,12cyclictest3974806-21fschecks_count12:20:163
386687699369,3cyclictest3912435-21latency_hist09:20:002
386687699362,9cyclictest3907968-21sed09:05:202
386687699362,9cyclictest3907968-21sed09:05:202
386687699362,11cyclictest3978723-21meminfo12:30:182
386687699361,9cyclictest3952474-21cut11:15:142
386687699361,8cyclictest3889540-21unixbench_multi08:10:282
386687699361,8cyclictest3889540-21unixbench_multi08:10:272
3866876993615,18cyclictest0-21swapper/210:40:132
3866876993613,20cyclictest3914104-21latency_hist09:25:002
386687699361,11cyclictest3977024-21memory12:25:192
386687699361,11cyclictest3874449-21awk07:30:012
386687699361,11cyclictest3874449-21awk07:30:012
386687699361,10cyclictest3951184-21grep11:10:222
3866869993627,4cyclictest0-21swapper/109:55:141
3866869993624,8cyclictest3923903-21grep09:50:281
3866865993629,3cyclictest0-21swapper/008:20:010
3866865993629,3cyclictest0-21swapper/008:20:000
3866865993628,4cyclictest3898749-21sendmail08:40:010
3866865993627,6cyclictest0-21swapper/010:25:010
3866865993626,7cyclictest3946935-21sendmail10:59:590
3866880993526,6cyclictest3951851-21cron11:13:013
3866880993526,6cyclictest3925074-21/usr/sbin/munin09:55:203
3866880993525,7cyclictest3880117-21/usr/sbin/munin07:45:153
3866880993525,7cyclictest3875116-21/usr/sbin/munin07:30:203
3866880993525,7cyclictest3867589-21/usr/sbin/munin07:10:203
3866880993518,12cyclictest3971770-21/usr/sbin/munin12:10:183
386687699359,2cyclictest0-21swapper/212:21:502
3866876993528,4cyclictest0-21swapper/211:09:592
386687699351,9cyclictest3964343-21cut11:50:162
3866876993512,4cyclictest3973625-21sendmail_mailqu12:15:232
386687699351,10cyclictest3875509-21ps07:30:272
3866869993526,5cyclictest0-21swapper/112:20:251
3866869993525,5cyclictest3912384-21cat09:20:001
3866869993524,5cyclictest0-21swapper/108:40:001
3866865993526,6cyclictest0-21swapper/011:50:000
3866865993525,6cyclictest3956676-21grep11:25:230
3866880993425,6cyclictest3959139-21/usr/sbin/munin11:35:133
3866880993419,10cyclictest3885200-21/usr/sbin/munin08:00:123
386687699349,3cyclictest0-21swapper/208:45:292
386687699347,6cyclictest204-21dbus-daemon10:19:372
386687699343,9cyclictest3868849-21df07:15:022
386687699341,11cyclictest3933423-21diskstats10:20:142
386687699341,11cyclictest3929987-21df10:10:132
386687699341,10cyclictest3981475-21awk12:40:002
3866869993426,4cyclictest0-21swapper/109:25:261
3866869993425,4cyclictest0-21swapper/108:20:171
3866865993421,10cyclictest0-21swapper/010:25:110
3866869993224,4cyclictest3873134-21fschecks_count07:25:201
3866869993224,4cyclictest3873134-21fschecks_count07:25:191
3866869993224,4cyclictest0-21swapper/110:25:221
3866869993222,6cyclictest0-21swapper/111:25:241
3866865993224,5cyclictest3874512-21latency_hist07:30:010
3866865993224,5cyclictest3874512-21latency_hist07:30:010
3866865993224,4cyclictest0-21swapper/010:35:000
386686999312,4cyclictest131rcu_preempt12:15:131
3866869993122,6cyclictest3870354-21grep07:15:271
3866869993118,8cyclictest0-21swapper/108:20:001
3866869993118,8cyclictest0-21swapper/108:20:001
3866865993124,4cyclictest0-21swapper/010:55:000
3866865993121,7cyclictest3886587-21debian-sa108:05:000
3866869993021,5cyclictest0-21swapper/111:10:001
3866869993021,5cyclictest0-21swapper/109:00:161
3866869993020,6cyclictest3934308-21tail10:20:241
3866869993020,6cyclictest0-21swapper/112:05:151
3866869993020,5cyclictest0-21swapper/108:50:171
3866869993020,4cyclictest0-21swapper/109:35:131
3866869993019,7cyclictest3908968-21sh09:10:001
3866869993019,7cyclictest3908968-21sh09:09:591
3866869993019,7cyclictest3901743-21grep08:45:291
3866869993019,6cyclictest0-21swapper/111:55:151
386686999301,7cyclictest131rcu_preempt07:40:151
3866865993023,4cyclictest0-21swapper/012:10:240
386687699291,12cyclictest3897596-21irqstats08:35:182
3866869992922,4cyclictest0-21swapper/111:35:151
3866869992921,4cyclictest0-21swapper/107:35:011
3866869992919,7cyclictest3968838-21grep12:00:251
3866865992922,4cyclictest0-21swapper/010:00:010
3866865992920,6cyclictest0-21swapper/007:10:150
3866869992821,4cyclictest0-21swapper/110:45:261
3866869992821,4cyclictest0-21swapper/110:45:251
3866869992817,7cyclictest3916984-21sort09:30:251
3866865992821,3cyclictest0-21swapper/010:09:590
3866865992820,4cyclictest0-21swapper/008:30:010
3866869992718,5cyclictest0-21swapper/110:45:001
3866869992718,4cyclictest0-21swapper/110:00:251
3866869992717,7cyclictest0-21swapper/112:14:591
3866869992717,5cyclictest0-21swapper/110:15:261
3866869992715,7cyclictest0-21swapper/108:35:011
386686599279,15cyclictest121ktimers/009:20:210
3866865992722,3cyclictest13550irq/42-30be0000.ethernet10:18:240
3866865992719,5cyclictest0-21swapper/011:54:590
3866865992719,4cyclictest0-21swapper/007:40:000
3866869992620,4cyclictest3870669-21kworker/u8:0+events_unbound07:20:421
3866869992619,4cyclictest0-21swapper/108:10:281
3866869992619,4cyclictest0-21swapper/108:10:271
3866869992618,5cyclictest3878094-21ls07:40:011
3866869992618,5cyclictest0-21swapper/111:40:161
3866869992617,6cyclictest3922209-21grep09:45:251
3866869992617,6cyclictest0-21swapper/110:30:141
3866869992616,7cyclictest3978003-21seq12:29:411
3866869992616,6cyclictest3958370-21unixbench_multi11:30:241
3866865992622,2cyclictest13550irq/42-30be0000.ethernet07:15:130
3866865992619,3cyclictest0-21swapper/012:15:260
3866865992617,6cyclictest0-21swapper/010:05:000
3866865992617,5cyclictest3905544-21cron09:00:000
3866865992616,6cyclictest0-21swapper/008:00:000
3866865992615,6cyclictest0-21swapper/012:00:000
386686999255,4cyclictest131rcu_preempt10:05:111
3866869992518,3cyclictest0-21swapper/107:10:191
3866869992517,4cyclictest0-21swapper/109:10:251
3866869992515,7cyclictest3894432-21ntp_states08:25:251
3866869992513,8cyclictest1-21systemd11:14:541
3866865992519,4cyclictest395-21dbus-daemon12:00:020
3866865992519,3cyclictest0-21swapper/007:45:250
3866865992516,5cyclictest0-21swapper/011:10:250
3866865992514,7cyclictest0-21swapper/011:20:010
386686599251,20cyclictest3912419-21lockfile-remove09:20:000
386686599251,20cyclictest0-21swapper/009:07:400
386686599251,20cyclictest0-21swapper/009:07:390
3866869992417,3cyclictest0-21swapper/111:45:151
3866869992415,6cyclictest3955463-21basename11:25:011
3866865992420,2cyclictest13550irq/42-30be0000.ethernet12:25:020
3866865992418,2cyclictest0-21swapper/008:45:180
3866865992417,3cyclictest0-21swapper/012:06:100
3866865992417,3cyclictest0-21swapper/011:04:110
3866865992415,5cyclictest0-21swapper/007:43:580
386686599241,20cyclictest0-21swapper/008:23:050
386686599241,19cyclictest0-21swapper/012:34:560
386686599241,19cyclictest0-21swapper/010:37:040
386686599241,19cyclictest0-21swapper/007:23:430
386686599240,20cyclictest0-21swapper/009:53:240
386686599240,20cyclictest0-21swapper/009:10:390
386686999238,4cyclictest221rcuc/112:30:231
3866869992317,3cyclictest0-21swapper/108:05:261
3866869992316,4cyclictest3939684-21head10:35:251
3866869992316,3cyclictest0-21swapper/111:19:541
3866869992315,6cyclictest3884199-21sed07:55:231
3866865992317,3cyclictest0-21swapper/009:40:200
3866865992316,4cyclictest3886957-21fschecks_count08:05:170
3866865992316,4cyclictest0-21swapper/008:30:270
386686599231,18cyclictest0-21swapper/010:47:290
386686599231,18cyclictest0-21swapper/010:47:290
386686599231,18cyclictest0-21swapper/009:34:560
386686599231,18cyclictest0-21swapper/007:31:260
3866865992311,10cyclictest151rcuc/010:10:120
386686599230,19cyclictest0-21swapper/008:51:430
3866869992215,4cyclictest3900438-21cron08:45:011
3866869992214,4cyclictest0-21swapper/111:00:261
3866869992213,6cyclictest0-21swapper/107:50:211
3866865992216,4cyclictest0-21swapper/008:10:240
3866865992216,4cyclictest0-21swapper/008:10:240
3866865992215,3cyclictest0-21swapper/012:39:550
3866865992212,7cyclictest204-21dbus-daemon11:09:540
386686599221,19cyclictest0-21swapper/007:50:220
386686599221,17cyclictest0-21swapper/008:42:410
386686599220,18cyclictest0-21swapper/009:01:270
3866869992114,3cyclictest0-21swapper/110:50:221
386686999209,8cyclictest3965109-21sed11:50:251
3866869992013,3cyclictest0-21swapper/108:59:221
3866869992011,6cyclictest3930086-21cut10:10:151
386686999181,12cyclictest3886598-21munin-run08:05:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional