You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 08:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Sun Apr 12, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3664613996638,25cyclictest3672401-21fschecks_time19:30:153
366460099571,3cyclictest0-21swapper/019:10:120
366460999511,17cyclictest32-21ksoftirqd/220:20:102
3664604995039,7cyclictest3760221-21munin-run23:45:011
366461399493,28cyclictest3684913-21ntp_states20:05:233
3664600994942,4cyclictest3648118-21kworker/u8:2+events_unbound19:37:210
3664613994633,9cyclictest391ktimers/321:10:143
3664609994537,4cyclictest3736120-21idleruntime-cro22:35:012
3664613994421,19cyclictest0-21swapper/321:20:253
366461399441,38cyclictest53-21kcompactd023:08:373
366461399431,36cyclictest3698316-21df_inode20:45:173
366461399431,35cyclictest3680922-21apt-get19:55:143
3664609994333,7cyclictest3670028-21kworker/u8:3+events_unbound19:37:252
3664604994333,7cyclictest3763602-21cron23:55:001
3664600994334,6cyclictest3768685-21kworker/u8:3+events_unbound00:32:560
3664600994334,6cyclictest3768685-21kworker/u8:3+events_unbound00:32:550
3664613994235,4cyclictest37363482sleep322:35:133
3664613994231,8cyclictest3778958-21sh00:40:003
366461399421,38cyclictest3724761-21/usr/sbin/munin22:00:183
366461399421,38cyclictest3714772-21/usr/sbin/munin21:30:233
366461399421,36cyclictest3768964-21apt-get00:10:123
366461399421,35cyclictest3751710-21apt-get23:20:133
3664613994129,9cyclictest3666425-21latency_hist19:15:023
366461399411,37cyclictest3774351-21/usr/sbin/munin00:25:173
366461399411,36cyclictest3765672-21df00:00:153
366461399411,35cyclictest3701805-21fschecks_time20:55:163
3664604994129,9cyclictest231ktimers/121:40:111
366460499411,37cyclictest3744628-21cron23:00:001
3664600994132,6cyclictest3687038-21unixbench_multi20:10:260
366461399403,34cyclictest3686443-21/usr/sbin/munin20:10:193
366461399403,34cyclictest1-21systemd19:22:203
366461399402,34cyclictest3667063-21apt-get19:15:113
366461399401,36cyclictest53-21kcompactd021:07:243
366461399401,36cyclictest3745226-21/usr/sbin/munin23:00:193
366461399401,36cyclictest3744463-21cron22:58:013
366461399401,36cyclictest3742761-21cron22:53:013
366461399401,36cyclictest3716444-21/usr/sbin/munin21:35:223
366461399401,36cyclictest3680019-21/usr/sbin/munin19:50:243
366461399401,35cyclictest3754405-21sed23:25:263
366461399401,35cyclictest3732995-21/usr/sbin/munin22:25:123
366461399401,34cyclictest3728443-21/usr/sbin/munin22:10:233
366461399401,34cyclictest3728443-21/usr/sbin/munin22:10:223
366461399401,34cyclictest3717936-21/usr/sbin/munin21:40:183
366461399401,34cyclictest3694679-21df_inode20:35:153
366461399401,33cyclictest3703422-21df21:00:143
366461399400,33cyclictest3767265-21apt-get00:05:113
3664604994030,5cyclictest0-21swapper/120:45:171
3664604994026,10cyclictest231ktimers/123:55:141
3664600994032,4cyclictest0-21swapper/020:40:000
366461399393,33cyclictest3761650-21cron23:47:013
366461399392,32cyclictest3693319-21/usr/sbin/munin20:30:223
3664613993916,20cyclictest0-21swapper/323:35:193
3664613993916,20cyclictest0-21swapper/321:26:323
366461399391,35cyclictest3776132-21/usr/sbin/munin00:30:203
366461399391,35cyclictest3776132-21/usr/sbin/munin00:30:203
366461399391,35cyclictest3734951-21/usr/sbin/munin22:30:193
366461399391,35cyclictest3709368-21/usr/sbin/munin21:15:223
366461399391,35cyclictest3709368-21/usr/sbin/munin21:15:213
366461399391,35cyclictest3696499-21apt-get20:40:123
366461399391,34cyclictest3700014-21/usr/sbin/munin20:50:153
366461399391,34cyclictest3689910-21meminfo20:20:193
366461399391,33cyclictest3772491-21df00:20:133
366461399391,33cyclictest3729830-21if_eth122:15:173
366461399391,33cyclictest3677968-21/usr/sbin/munin19:45:193
3664600993932,4cyclictest3768264-21unixbench_multi00:05:250
366461399383,32cyclictest3682976-21irqstats20:00:183
366461399383,18cyclictest3755225-21diskstats23:30:143
366461399382,33cyclictest3770970-21/usr/sbin/munin00:15:173
366461399382,33cyclictest3762143-21apt-get23:50:133
366461399381,34cyclictest3759611-21timerandwakeup23:40:243
366461399381,34cyclictest3738060-21/usr/sbin/munin22:40:253
366461399381,34cyclictest3721236-21/usr/sbin/munin21:50:163
366461399381,32cyclictest3750084-21cstates23:15:133
366461399381,32cyclictest3748519-21/usr/sbin/munin23:10:153
366461399381,32cyclictest3739865-21/usr/sbin/munin22:45:143
366461399381,32cyclictest3722808-21df21:55:143
366461399381,31cyclictest3687743-21apt-get20:15:133
3664609993832,4cyclictest3704555-21kworker/u8:0+events_unbound21:07:392
3664609993825,9cyclictest3726334-21grep22:05:182
3664600993828,6cyclictest3671510-21unixbench_multi19:25:260
366461399371,33cyclictest3731245-21apt-get22:20:113
366461399371,33cyclictest3719385-21/usr/sbin/munin21:45:143
366461399371,33cyclictest3692189-21/usr/sbin/munin20:25:273
366461399371,33cyclictest3692189-21/usr/sbin/munin20:25:273
366461399371,33cyclictest3673654-21apt-get19:35:063
366461399371,31cyclictest3670598-21/usr/sbin/munin19:25:153
366461399370,33cyclictest53-21kcompactd023:55:023
3664600993730,4cyclictest3744646-21sendmail23:00:010
3664600993728,6cyclictest3754410-21unixbench_multi23:25:260
366461399361,32cyclictest3675928-21df_inode19:40:133
3664609993625,7cyclictest3770429-21sh00:15:002
3664604993626,6cyclictest0-21swapper/122:40:001
3664604993625,7cyclictest3695606-21grep20:35:271
3664600993630,4cyclictest3704555-21kworker/u8:0+events_unbound21:12:400
3664600993628,5cyclictest3731000-21munin-run22:19:590
366461399351,31cyclictest3726640-21/usr/sbin/munin22:05:213
3664609993529,4cyclictest3742798-21kworker/u8:0+events_unbound23:12:492
3664609993528,5cyclictest3704555-21kworker/u8:0+events_unbound21:52:422
3664609993527,4cyclictest0-21swapper/219:35:002
3664609993525,7cyclictest3760207-21sh23:45:002
3664604993526,6cyclictest3766612-21grep00:00:271
3664604993525,7cyclictest3756572-21sh23:34:591
3664604993525,5cyclictest3711566-21grep21:20:261
3664600993525,6cyclictest0-21swapper/022:40:000
3664609993417,7cyclictest3680926-21kworker/u8:3+events_unbound20:10:412
3664604993426,4cyclictest0-21swapper/120:10:271
3664604993424,6cyclictest0-21swapper/121:05:011
3664600993426,4cyclictest3717341-21cron21:40:010
3664600993424,6cyclictest0-21swapper/023:34:590
3664600993424,6cyclictest0-21swapper/020:50:160
3664600993422,8cyclictest3765328-21sendmail00:00:000
3664609993327,4cyclictest3775378-21kworker/u8:1+events_unbound00:37:552
3664609993326,3cyclictest0-21swapper/221:35:262
3664609993318,6cyclictest3711740-21kworker/u8:2+events_unbound22:19:142
3664604993324,4cyclictest0-21swapper/100:32:551
3664604993324,4cyclictest0-21swapper/100:32:551
366460099333,6cyclictest131rcu_preempt22:05:140
3664600993327,4cyclictest3749593-21kworker/u8:1+events_unbound00:17:570
3664600993326,5cyclictest3648118-21kworker/u8:2+events_unbound19:22:220
3664600993325,4cyclictest0-21swapper/023:35:230
3664609993226,3cyclictest3741240-21kthreadd22:50:002
3664609993225,4cyclictest3724165-21cron22:00:002
3664609993223,4cyclictest0-21swapper/219:15:022
3664609993212,7cyclictest131rcu_preempt21:30:122
3664604993224,4cyclictest0-21swapper/121:10:271
3664600993226,3cyclictest0-21swapper/022:00:010
3664600993225,4cyclictest3706496-21munin-plugin-st21:10:010
3664600993225,3cyclictest0-21swapper/021:25:270
3664600993224,5cyclictest3684098-21cron20:05:010
3664600993222,6cyclictest3747600-21unixbench_multi23:05:250
3664600993222,6cyclictest0-21swapper/021:30:260
3664600993222,6cyclictest0-21swapper/019:55:270
3664609993124,4cyclictest0-21swapper/220:15:252
3664609993123,4cyclictest0-21swapper/220:05:012
3664609993121,6cyclictest0-21swapper/223:50:002
3664609993110,13cyclictest131rcu_preempt21:15:002
366460999310,16cyclictest0-21swapper/221:40:132
3664604993123,4cyclictest0-21swapper/122:30:001
3664604993123,4cyclictest0-21swapper/122:15:261
366460099319,3cyclictest37193582ls21:45:140
366460099312,5cyclictest131rcu_preempt23:10:110
3664600993124,4cyclictest0-21swapper/020:30:270
3664600993121,6cyclictest0-21swapper/020:10:000
3664600993120,7cyclictest3668070-21unixbench_multi19:15:240
3664600993112,10cyclictest11-21ksoftirqd/020:20:110
366460999308,12cyclictest131rcu_preempt19:55:002
366460999302,4cyclictest131rcu_preempt23:15:112
3664609993023,5cyclictest3766684-21kworker/u8:0+events_unbound00:01:412
3664609993023,5cyclictest3704555-21kworker/u8:0+events_unbound21:02:382
3664609993022,4cyclictest0-21swapper/222:50:252
3664609993021,5cyclictest3693927-21unixbench_singl20:30:282
3664604993019,8cyclictest3683695-21grep20:00:251
3664604993019,6cyclictest3735656-21grep22:30:261
3664600993022,4cyclictest0-21swapper/020:55:250
3664600993012,4cyclictest151rcuc/021:45:010
3664609992923,3cyclictest0-21swapper/223:24:592
3664609992922,5cyclictest3734277-21kworker/u8:0+events_unbound22:40:122
3664609992921,5cyclictest3775563-21idleruntime-cro00:30:002
3664609992921,4cyclictest0-21swapper/200:15:152
366460999290,6cyclictest3701356-21cron20:55:012
3664604992916,9cyclictest3713983-21latency_hist21:30:011
3664600992921,4cyclictest0-21swapper/023:00:150
3664600992920,6cyclictest3699185-21grep20:45:280
3664609992821,5cyclictest3749593-21kworker/u8:1+events_unbound23:27:542
3664609992821,3cyclictest0-21swapper/223:00:262
3664609992820,4cyclictest0-21swapper/221:45:172
3664609992817,7cyclictest0-21swapper/221:25:162
3664609992811,6cyclictest32-21ksoftirqd/200:05:242
3664604992820,4cyclictest0-21swapper/119:45:211
3664604992817,8cyclictest3773073-21find00:20:211
3664600992820,4cyclictest0-21swapper/000:20:230
3664600992816,7cyclictest204-21dbus-daemon19:52:260
366460099280,4cyclictest131rcu_preempt23:50:130
3664609992720,5cyclictest3677184-21kworker/u8:0+events_unbound19:48:572
3664609992720,4cyclictest0-21swapper/220:25:172
3664609992720,4cyclictest0-21swapper/220:25:162
3664609992719,5cyclictest3687308-21kworker/u8:1+events_unbound20:56:422
3664609992717,7cyclictest3731399-21tune2fs22:20:152
3664609992713,8cyclictest131rcu_preempt22:35:182
366460999271,13cyclictest0-21swapper/223:30:182
3664604992721,3cyclictest3751461-21kthreadd23:20:011
3664604992720,4cyclictest0-21swapper/119:25:021
3664604992717,6cyclictest3697554-21grep20:40:261
3664604992716,8cyclictest3758513-21sendmail23:40:011
3664604992716,8cyclictest231ktimers/123:25:121
3664600992720,4cyclictest3722483-21munin-plugin-st21:55:000
3664600992720,4cyclictest3704789-21cron21:05:010
3664600992719,4cyclictest0-21swapper/000:15:010
3664600992717,6cyclictest3761441-21grep23:45:250
3664600992712,11cyclictest121ktimers/022:10:120
3664600992712,11cyclictest121ktimers/022:10:120
366460999265,14cyclictest32-21ksoftirqd/222:05:002
3664609992620,4cyclictest3772128-21kworker/u8:0+events_unbound00:23:152
3664609992619,4cyclictest0-21swapper/220:35:262
366460999261,7cyclictest3666924-21fschecks_count19:15:052
366460999261,7cyclictest32-21ksoftirqd/220:45:002
3664609992613,6cyclictest3727416-21kworker/u8:3+events_unbound22:26:212
3664609992613,5cyclictest3650152-21kworker/u8:0+events_unbound19:20:582
366460999261,12cyclictest0-21swapper/222:12:422
366460999261,12cyclictest0-21swapper/222:12:422
3664604992618,5cyclictest3675572-21sh19:40:001
3664604992618,4cyclictest0-21swapper/122:40:161
3664604992617,4cyclictest0-21swapper/121:55:171
3664600992617,7cyclictest121ktimers/020:40:060
3664600992615,8cyclictest3732094-21wc22:20:240
366460099261,4cyclictest131rcu_preempt22:40:110
3664609992519,3cyclictest0-21swapper/220:05:172
3664609992519,3cyclictest0-21swapper/219:55:172
3664609992513,5cyclictest3709938-21kworker/u8:3+events_unbound21:17:402
3664609992513,5cyclictest3709938-21kworker/u8:3+events_unbound21:17:402
366460999251,2cyclictest3743167-21apt-get22:55:122
366460999250,5cyclictest0-21swapper/200:30:212
366460999250,5cyclictest0-21swapper/200:30:202
3664604992518,4cyclictest0-21swapper/122:00:241
3664604992518,4cyclictest0-21swapper/119:35:001
3664604992515,7cyclictest3719840-21ntpq21:45:221
3664600992516,7cyclictest3766107-21ntp_kernel_pll_00:00:210
3664600992514,3cyclictest11-21ksoftirqd/000:35:260
366460099251,20cyclictest0-21swapper/022:48:540
366460099251,20cyclictest0-21swapper/020:17:450
366460099250,4cyclictest131rcu_preempt22:50:130
3664609992418,4cyclictest3759899-21kworker/u8:3+events_unbound23:55:472
3664609992418,4cyclictest3759899-21kworker/u8:3-events_freezable_power_23:50:162
3664609992418,3cyclictest0-21swapper/219:45:012
3664604992417,3cyclictest0-21swapper/121:35:251
3664604992416,4cyclictest0-21swapper/100:05:171
3664604992410,9cyclictest231ktimers/119:10:111
3664600992418,3cyclictest0-21swapper/022:00:150
3664600992417,4cyclictest3672409-21fschecks_time19:30:150
3664600992417,3cyclictest0-21swapper/000:25:400
3664600992414,6cyclictest0-21swapper/022:25:240
3664600992414,6cyclictest0-21swapper/019:40:240
366460099241,19cyclictest0-21swapper/023:18:020
366460099241,19cyclictest0-21swapper/019:46:330
366460999238,5cyclictest131rcu_preempt19:25:252
3664609992312,7cyclictest0-21swapper/221:24:592
366460499231,8cyclictest3679782-21ntp_states19:50:221
3664604992317,3cyclictest0-21swapper/122:50:231
3664604992316,4cyclictest3706490-21kthreadd21:10:011
3664604992315,4cyclictest0-21swapper/123:10:251
3664604992313,7cyclictest3708994-21cut21:15:181
3664604992313,7cyclictest3708994-21cut21:15:171
3664604992312,6cyclictest0-21swapper/119:40:251
3664600992318,3cyclictest0-21swapper/021:15:050
3664600992318,3cyclictest0-21swapper/021:15:050
3664600992317,4cyclictest121ktimers/021:20:110
3664600992316,4cyclictest0-21swapper/023:20:260
366460099231,18cyclictest0-21swapper/023:41:400
366460099231,18cyclictest0-21swapper/020:25:310
366460099231,18cyclictest0-21swapper/020:25:300
3664604992215,5cyclictest231ktimers/121:30:171
3664604992215,5cyclictest231ktimers/120:55:241
3664604992215,5cyclictest231ktimers/100:10:101
3664604992215,4cyclictest3761973-21latency_hist23:50:001
3664604992214,5cyclictest3753174-21sh23:24:591
3664604992214,5cyclictest0-21swapper/122:45:221
3664604992212,7cyclictest1-21systemd20:17:311
366460099221,3cyclictest3734646-21apt-get22:30:140
3664604992115,3cyclictest0-21swapper/120:30:001
3664604992115,3cyclictest0-21swapper/120:30:001
3664604992114,5cyclictest231ktimers/100:35:131
3664604992114,3cyclictest0-21swapper/100:15:241
366460499211,14cyclictest3671530-21unixbench_singl19:25:261
366460499211,13cyclictest0-21swapper/120:22:081
366460499209,6cyclictest3721189-21fschecks_time21:50:171
3664604992015,3cyclictest645-21runrttasks22:10:201
3664604992015,3cyclictest645-21runrttasks22:10:191
3664604992014,4cyclictest3680918-21/usr/sbin/munin19:55:161
3664604992014,3cyclictest3685334-21grep20:05:261
3664604992014,3cyclictest0-21swapper/123:10:011
3664604992014,3cyclictest0-21swapper/122:20:251
3664604992014,3cyclictest0-21swapper/122:05:251
3664604992013,4cyclictest231ktimers/100:25:351
3664604992010,7cyclictest3745031-21cut23:00:151
366460499199,7cyclictest3699896-21apt-get20:50:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional