You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 21:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Jan 07, 2026 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
413899995824,4cyclictest0-21swapper/311:20:263
413891995739,15cyclictest495490-21munin-plugin-st11:05:001
413899995522,6cyclictest0-21swapper/312:30:243
413891995329,20cyclictest457828-21fschecks_time09:15:181
413899995121,4cyclictest0-21swapper/311:10:163
413886995117,10cyclictest131rcu_preempt09:40:120
413899995042,5cyclictest1-21systemd07:33:483
413899995042,5cyclictest1-21systemd07:33:483
413899995024,8cyclictest465500-21grep09:35:263
413895995037,9cyclictest311ktimers/210:00:122
413891995041,5cyclictest521596-21fschecks_time12:20:161
413895994935,10cyclictest311ktimers/211:40:092
413891994938,5cyclictest483293-21munin-plugin-st10:30:001
41389199471,13cyclictest480216-21df10:20:161
413891994619,23cyclictest0-21swapper/108:50:251
413895994536,5cyclictest502338-21munin-plugin-st11:25:002
413895994535,6cyclictest460417-21grep09:20:242
413891994437,4cyclictest457399-21cron09:15:001
413891994420,20cyclictest0-21swapper/110:40:181
41389199440,3cyclictest0-21swapper/111:50:121
413899994334,5cyclictest431313-21unixbench_singl07:55:273
413886994332,7cyclictest514311-21sendmail11:59:590
413899994232,6cyclictest445432-21idleruntime-cro08:40:013
41389199421,9cyclictest492282-21munin-node10:55:121
41389199421,7cyclictest454781-21/usr/sbin/munin09:05:201
41389199421,3cyclictest0-21swapper/112:35:121
413899994133,5cyclictest454001-21idleruntime-cro09:05:013
413899994128,9cyclictest459139-21sendmail09:20:003
41389199413,33cyclictest492227-21cat10:55:021
41389199411,9cyclictest520187-21/usr/sbin/munin12:15:201
41389199411,9cyclictest520187-21/usr/sbin/munin12:15:191
41389199411,9cyclictest422300-21/usr/sbin/munin07:30:261
41389199411,9cyclictest422300-21/usr/sbin/munin07:30:251
41389199411,8cyclictest501425-21/usr/sbin/munin11:20:221
41389199411,3cyclictest0-21swapper/110:15:131
41389199411,35cyclictest523205-21df_inode12:25:151
41389199411,12cyclictest438879-21/usr/sbin/munin08:20:151
41389199411,12cyclictest420081-21/usr/sbin/munin07:25:181
41389199411,10cyclictest445806-21forks08:40:161
413899994032,4cyclictest0-21swapper/310:05:183
413899994030,7cyclictest520724-21grep12:15:253
413899994030,7cyclictest520724-21grep12:15:243
413895994033,5cyclictest428068-21kworker/u8:2+events_unbound07:58:542
413895994028,8cyclictest438587-21idleruntime-cro08:20:012
41389199401,9cyclictest470393-21/usr/sbin/munin09:50:241
41389199401,6cyclictest505968-21/usr/sbin/munin11:35:241
41389199401,3cyclictest0-21swapper/112:05:131
41389199401,3cyclictest0-21swapper/112:05:131
41389199401,3cyclictest0-21swapper/111:55:111
41389199401,3cyclictest0-21swapper/110:10:131
413891994013,24cyclictest474742-21latency_hist10:05:001
41389199400,3cyclictest0-21swapper/107:15:131
413895993933,4cyclictest527792-21kworker/u8:3+events_unbound12:39:402
413895993920,7cyclictest131rcu_preempt07:15:132
41389199391,9cyclictest467996-21/usr/sbin/munin09:45:121
41389199391,8cyclictest428470-21/usr/sbin/munin07:50:161
41389199391,8cyclictest428470-21/usr/sbin/munin07:50:161
41389199391,7cyclictest484083-21ntp_states10:30:211
41389199391,7cyclictest464747-21/usr/sbin/munin09:35:161
41389199391,5cyclictest525134-21/usr/sbin/munin12:30:161
41389199391,4cyclictest514534-21/usr/sbin/munin12:00:261
41389199391,3cyclictest0-21swapper/111:15:121
41389199391,3cyclictest0-21swapper/108:45:121
41389199391,3cyclictest0-21swapper/108:30:121
41389199391,32cyclictest435555-21cut08:10:161
41389199391,10cyclictest451238-21meminfo08:55:201
413886993929,6cyclictest482837-21grep10:25:260
413899993830,4cyclictest497194-21idleruntime-cro11:10:003
413899993827,7cyclictest505267-21grep11:30:253
413895993828,7cyclictest311ktimers/208:20:142
413895993816,7cyclictest131rcu_preempt11:05:132
413895993816,7cyclictest131rcu_preempt10:50:122
41389199382,9cyclictest505226-21sed11:30:241
41389199382,5cyclictest425497-21ntp_states07:40:231
41389199382,5cyclictest425497-21ntp_states07:40:231
41389199382,4cyclictest475419-21meminfo10:05:201
41389199381,9cyclictest441382-21/usr/sbin/munin08:25:251
41389199381,8cyclictest629-21snmpd09:33:351
41389199381,3cyclictest0-21swapper/111:10:141
41389199381,3cyclictest0-21swapper/109:40:121
41389199381,33cyclictest415515-21cron07:15:001
41389199381,2cyclictest0-21swapper/107:45:101
41389199381,2cyclictest0-21swapper/107:45:091
41389199380,3cyclictest0-21swapper/111:45:131
413895993731,4cyclictest519252-21kworker/u8:1+events_unbound12:18:422
413895993731,4cyclictest519252-21kworker/u8:1+events_unbound12:18:422
413895993726,6cyclictest0-21swapper/207:30:262
413895993726,6cyclictest0-21swapper/207:30:262
41389199377,27cyclictest0-21swapper/110:35:221
41389199372,30cyclictest472805-21munin-run10:00:011
41389199372,30cyclictest472805-21munin-run10:00:001
41389199371,6cyclictest418534-21memory07:20:201
41389199371,4cyclictest518478-21/usr/sbin/munin12:10:181
41389199371,4cyclictest518478-21/usr/sbin/munin12:10:181
41389199371,4cyclictest502674-21/usr/sbin/munin11:25:141
41389199371,4cyclictest489849-21/usr/sbin/munin10:45:261
41389199371,3cyclictest434668-21/usr/sbin/munin08:05:231
41389199371,3cyclictest0-21swapper/109:25:151
41389199371,33cyclictest629-21snmpd07:38:131
41389199371,33cyclictest629-21snmpd07:38:131
41389199371,10cyclictest443948-21apt-get08:35:111
41389199370,3cyclictest0-21swapper/111:05:131
413886993723,10cyclictest121ktimers/008:25:130
413899993627,6cyclictest0-21swapper/310:55:273
413899993625,7cyclictest518943-21grep12:10:243
413899993625,7cyclictest518943-21grep12:10:243
41389199362,31cyclictest460395-21/usr/sbin/munin09:20:251
413886993628,4cyclictest0-21swapper/012:10:000
413886993628,4cyclictest0-21swapper/012:09:590
413899993524,6cyclictest0-21swapper/311:55:263
413895993528,5cyclictest450312-21kworker/u8:2+events_unbound08:54:022
413895993527,4cyclictest311ktimers/210:30:142
413895993525,6cyclictest524151-21grep12:25:242
41389199351,7cyclictest508042-21cut11:40:161
41389199351,3cyclictest452510-21/usr/sbin/munin09:00:251
41389199350,4cyclictest431983-21apt-get08:00:151
41389199350,3cyclictest0-21swapper/107:55:131
413886993527,4cyclictest0-21swapper/008:55:260
413899993424,6cyclictest0-21swapper/311:35:223
41389599349,12cyclictest131rcu_preempt11:50:252
413895993428,4cyclictest470919-21kworker/u8:0+events_unbound10:08:412
413895993423,7cyclictest427688-21grep07:45:232
413895993423,7cyclictest427688-21grep07:45:232
413886993428,4cyclictest433457-21sh08:05:010
413886993427,3cyclictest0-21swapper/007:50:010
413886993427,3cyclictest0-21swapper/007:50:010
413886993423,7cyclictest496701-21grep11:05:240
41388699341,30cyclictest0-21swapper/011:30:100
413899993326,4cyclictest447158-21munin-plugin-st08:45:013
413899993324,6cyclictest510406-21grep11:45:263
413895993325,4cyclictest0-21swapper/207:35:182
413895993325,4cyclictest0-21swapper/207:35:182
413895993323,7cyclictest469386-21sh09:50:012
413895993318,6cyclictest517415-21kworker/u8:0+events_unbound12:20:182
413895993311,6cyclictest131rcu_preempt12:30:122
413886993325,4cyclictest0-21swapper/011:25:260
413886993321,8cyclictest507031-21grep11:35:220
41388699331,29cyclictest0-21swapper/007:30:140
41388699331,29cyclictest0-21swapper/007:30:140
41388699331,11cyclictest4603622unixbench_multi09:20:240
413899993224,4cyclictest0-21swapper/312:39:593
413899993224,4cyclictest0-21swapper/312:25:253
413899993224,4cyclictest0-21swapper/310:10:243
41389599329,11cyclictest131rcu_preempt10:45:142
413895993211,11cyclictest32-21ksoftirqd/208:00:262
413886993224,4cyclictest0-21swapper/009:10:260
413886993223,6cyclictest521027-21kworker/u8:2+events_unbound12:24:370
413886993222,7cyclictest420827-21grep07:25:260
413886993222,6cyclictest0-21swapper/012:19:590
413886993222,6cyclictest0-21swapper/012:19:590
413886993221,7cyclictest417366-21grep07:15:260
413886993221,6cyclictest0-21swapper/010:10:150
413899993122,6cyclictest0-21swapper/311:45:003
413895993124,4cyclictest0-21swapper/207:15:012
413895993124,3cyclictest0-21swapper/211:40:012
413895993123,4cyclictest0-21swapper/208:35:252
413895993113,4cyclictest131rcu_preempt11:00:122
413886993125,3cyclictest0-21swapper/008:40:010
413886993121,6cyclictest0-21swapper/009:35:160
413886993120,7cyclictest500112-21grep11:15:240
413899993022,4cyclictest0-21swapper/310:55:013
413899993022,4cyclictest0-21swapper/307:43:483
413899993022,4cyclictest0-21swapper/307:43:483
413899993021,6cyclictest485612-21tune2fs10:35:163
41389599308,13cyclictest131rcu_preempt10:20:162
413895993024,4cyclictest453652-21kworker/u8:0+events_unbound09:14:072
413895993024,3cyclictest0-21swapper/209:25:262
413895993023,5cyclictest467437-21kworker/u8:1+events_unbound09:44:112
413895993022,6cyclictest453652-21kworker/u8:0+events_unbound09:02:262
413895993021,7cyclictest498568-21kworker/u8:1+events_unbound11:30:252
413895993019,7cyclictest443708-21cron08:35:002
413895993014,6cyclictest32-21ksoftirqd/210:25:262
413886993020,6cyclictest0-21swapper/009:20:000
413886993019,8cyclictest121ktimers/007:10:150
413899992919,7cyclictest469802-21fschecks_time09:50:153
41389599298,13cyclictest32-21ksoftirqd/210:35:142
413895992923,4cyclictest511283-21kworker/u8:1+events_unbound12:04:362
413895992922,4cyclictest479396-21grep10:15:242
413895992921,6cyclictest465660-21kworker/u8:0+events_unbound09:36:432
413895992921,4cyclictest0-21swapper/211:55:262
413886992921,4cyclictest0-21swapper/012:39:590
413886992921,4cyclictest0-21swapper/010:45:260
413886992920,5cyclictest483702-21grep10:30:160
413899992821,3cyclictest0-21swapper/312:20:203
413899992820,5cyclictest478590-21grep10:15:153
413899992818,7cyclictest450080-21grep08:50:243
413899992818,6cyclictest419145-21grep07:20:253
413899992815,9cyclictest204-21dbus-daemon08:28:573
41389599288,4cyclictest131rcu_preempt09:15:182
413895992820,4cyclictest493759-21kthreadd11:00:012
413895992820,4cyclictest0-21swapper/207:20:252
413895992819,6cyclictest509560-21fschecks_count11:45:162
413886992821,4cyclictest448852-21apt08:50:000
413886992817,8cyclictest121ktimers/007:50:140
413886992817,8cyclictest121ktimers/007:50:130
413899992719,5cyclictest0-21swapper/312:00:243
413899992719,4cyclictest0-21swapper/309:45:233
413899992717,6cyclictest0-21swapper/311:25:263
41389599278,12cyclictest32-21ksoftirqd/208:55:212
41389599276,13cyclictest131rcu_preempt08:40:182
41389599275,14cyclictest32-21ksoftirqd/208:45:252
413895992721,4cyclictest498568-21kworker/u8:1+events_unbound11:11:412
413886992720,4cyclictest0-21swapper/010:10:010
413886992717,6cyclictest0-21swapper/007:25:000
413899992620,3cyclictest0-21swapper/307:15:033
413899992619,5cyclictest208-21systemd-logind09:32:413
413899992619,4cyclictest424221-21tail07:35:273
413899992619,4cyclictest424221-21tail07:35:273
413899992619,4cyclictest0-21swapper/311:00:253
413899992618,5cyclictest0-21swapper/308:15:283
413899992618,4cyclictest474286-21grep10:00:263
413899992616,6cyclictest436436-21grep08:10:263
413899992613,9cyclictest391ktimers/309:10:143
41389599268,7cyclictest32-21ksoftirqd/207:50:272
41389599268,7cyclictest32-21ksoftirqd/207:50:272
41389599266,13cyclictest32-21ksoftirqd/209:50:152
41389599265,4cyclictest131rcu_preempt10:00:002
41389599265,4cyclictest131rcu_preempt10:00:002
41389599265,4cyclictest0-21swapper/209:05:142
413895992618,6cyclictest484657-21kworker/u8:0+events_unbound10:44:422
413895992617,7cyclictest515701-21kworker/u8:2+events_unbound12:08:402
413895992617,7cyclictest515701-21kworker/u8:2+events_unbound12:08:402
413895992614,4cyclictest410912-21kworker/2:1-events07:25:172
413886992619,4cyclictest523274-21grep12:25:160
413886992619,4cyclictest455769-21latency_hist09:10:010
413886992617,6cyclictest486498-21unixbench_singl10:35:250
41389599257,5cyclictest131rcu_preempt08:25:252
413895992518,5cyclictest515701-21kworker/u8:2+events_unbound12:10:142
413895992518,5cyclictest515701-21kworker/u8:2+events_unbound12:10:132
413895992518,5cyclictest477121-21kworker/u8:2+events_unbound10:10:192
413895992517,5cyclictest0-21swapper/209:30:152
41389599251,12cyclictest0-21swapper/208:05:122
413886992521,2cyclictest13550irq/42-30be0000.ethernet11:00:130
413886992519,4cyclictest480105-21/usr/sbin/munin10:20:160
413886992519,3cyclictest0-21swapper/007:35:180
413886992519,3cyclictest0-21swapper/007:35:170
413886992518,4cyclictest516014-21sh12:04:590
413886992518,4cyclictest0-21swapper/007:40:260
413886992518,4cyclictest0-21swapper/007:40:260
413886992517,3cyclictest0-21swapper/010:00:010
413886992517,3cyclictest0-21swapper/010:00:010
413886992515,6cyclictest0-21swapper/010:15:250
413886992515,5cyclictest0-21swapper/011:55:000
41388699251,21cyclictest4622852sleep009:26:590
413886992511,11cyclictest398159-1kworker/0:1H+mmc_complete08:20:000
413899992419,2cyclictest0-21swapper/308:20:123
413899992417,4cyclictest0-21swapper/312:05:253
413899992417,4cyclictest0-21swapper/312:05:243
413899992416,6cyclictest391ktimers/310:35:013
413899992415,6cyclictest414903-21cat07:10:273
41389599249,3cyclictest131rcu_preempt11:20:002
41389599247,6cyclictest32-21ksoftirqd/207:40:272
41389599247,6cyclictest32-21ksoftirqd/207:40:262
41389599243,11cyclictest0-21swapper/208:13:542
41388699243,19cyclictest0-21swapper/008:30:460
413886992419,3cyclictest13550irq/42-30be0000.ethernet08:40:140
413886992417,4cyclictest0-21swapper/011:45:260
413886992417,4cyclictest0-21swapper/009:50:000
413886992417,3cyclictest0-21swapper/008:05:200
413886992416,5cyclictest0-21swapper/010:50:260
413886992416,3cyclictest0-21swapper/008:50:190
413886992414,6cyclictest0-21swapper/009:00:250
41388699241,20cyclictest508803-21taskset11:40:430
413899992317,3cyclictest0-21swapper/309:25:003
413899992316,3cyclictest0-21swapper/307:25:223
41388699232,10cyclictest0-21swapper/010:43:500
413886992318,3cyclictest13550irq/42-30be0000.ethernet10:55:160
413886992317,3cyclictest13850irq/36-mmc111:15:000
413886992317,2cyclictest0-21swapper/012:10:230
413886992317,2cyclictest0-21swapper/012:10:230
41388699231,18cyclictest0-21swapper/012:31:140
41388699231,18cyclictest0-21swapper/008:12:060
413899992216,3cyclictest0-21swapper/308:45:223
413899992215,4cyclictest0-21swapper/308:30:253
41388699221,9cyclictest462884-21df09:30:140
413886992214,6cyclictest121ktimers/010:00:060
413886992214,5cyclictest0-21swapper/008:20:160
41388699221,17cyclictest0-21swapper/011:24:130
413899992115,3cyclictest0-21swapper/311:15:123
413899992114,3cyclictest0-21swapper/309:40:253
413899992111,6cyclictest0-21swapper/307:45:233
413899992111,6cyclictest0-21swapper/307:45:233
413899992014,4cyclictest391ktimers/310:26:323
413899992014,3cyclictest0-21swapper/310:45:223
413899992014,3cyclictest0-21swapper/307:50:153
413899992014,3cyclictest0-21swapper/307:50:143
413899992013,5cyclictest391ktimers/311:50:133
413899992013,5cyclictest0-21swapper/309:25:233
413899992010,7cyclictest455884-21wc09:10:023
41389999200,18cyclictest391ktimers/308:05:123
41388699201,17cyclictest0-21swapper/009:50:480
413899991913,3cyclictest0-21swapper/310:25:013
413899991913,3cyclictest0-21swapper/308:00:143
413899991811,5cyclictest391ktimers/308:55:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional