You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 06:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sat Sep 05, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2415750996525,27cyclictest0-21swapper/319:15:273
241575099491,30cyclictest2504810-21meminfo23:25:203
241575099490,41cyclictest0-21swapper/319:50:113
241575099490,41cyclictest0-21swapper/319:50:113
241575099490,40cyclictest0-21swapper/320:10:153
241575099490,35cyclictest2493502-21latency_hist22:55:003
2415743994941,5cyclictest2472587-21unixbench_singl21:50:241
241575099461,27cyclictest2486645-21latency_hist22:35:013
2415750994539,3cyclictest2447102-21latency_hist20:40:013
2415743994530,11cyclictest231ktimers/122:15:111
2415750994422,19cyclictest2469124-21unixbench_multi21:40:243
241575099441,38cyclictest2463262-21/usr/sbin/munin21:25:173
2415746994429,10cyclictest311ktimers/200:10:122
2415743994433,8cyclictest2511209-21latency_hist23:45:011
2415750994327,12cyclictest0-21swapper/321:50:003
2415750994316,24cyclictest2515117-21if_eth123:55:163
241575099431,39cyclictest2512422-21/usr/sbin/munin23:45:243
241575099431,37cyclictest2493692-21/usr/sbin/munin22:55:103
2415743994333,7cyclictest2461063-21apt-key21:20:021
2415743994329,11cyclictest231ktimers/122:40:101
2415750994218,20cyclictest2447644-21fschecks_time20:40:163
241575099421,38cyclictest2435627-21/usr/sbin/munin20:05:183
241575099421,36cyclictest2450837-21apt-get20:50:133
2415746994230,5cyclictest2489942-21kworker/u8:3+events_unbound22:52:062
2415743994235,4cyclictest2443576-21latency_hist20:30:011
2415739994234,5cyclictest2454028-21sendmail21:00:000
241575099411,35cyclictest2513095-21apt-get23:50:123
241575099411,35cyclictest2481929-21/usr/sbin/munin22:20:173
241575099411,35cyclictest2423564-21/usr/sbin/munin19:30:153
241575099411,35cyclictest2416033-21/usr/sbin/munin19:10:143
241575099411,35cyclictest2416033-21/usr/sbin/munin19:10:143
241575099411,34cyclictest2507764-21apt-get23:35:113
241575099411,34cyclictest2471521-21apt-get21:50:113
2415743994134,5cyclictest2422914-21kworker/u8:2+events_unbound19:30:421
241575099403,32cyclictest2523677-21diskstats00:20:173
241575099403,32cyclictest2480150-21df22:15:133
241575099402,33cyclictest2432228-21/usr/sbin/munin19:55:183
241575099401,36cyclictest2526009-21/usr/sbin/munin00:25:243
241575099401,36cyclictest2501612-21/usr/sbin/munin23:15:223
241575099401,36cyclictest2439350-21rt-features20:15:213
241575099401,35cyclictest2497667-21/usr/sbin/munin23:05:173
241575099401,35cyclictest2443444-21sshd20:28:333
241575099401,34cyclictest2518433-21apt-get00:05:123
241575099401,34cyclictest2426885-21/usr/sbin/munin19:40:153
241575099401,34cyclictest2426885-21/usr/sbin/munin19:40:153
241575099401,34cyclictest2421749-21/usr/sbin/munin19:25:153
241575099401,33cyclictest2457663-21apt-get21:10:123
241575099401,33cyclictest2425371-21/usr/sbin/munin19:35:183
2415750994010,27cyclictest2476644-21/usr/sbin/munin22:05:163
2415746994031,4cyclictest0-21swapper/219:50:162
2415746994031,4cyclictest0-21swapper/219:50:162
241575099392,34cyclictest2529944-21cron00:38:003
241575099391,35cyclictest2510001-21/usr/sbin/munin23:40:203
241575099391,35cyclictest2500007-21/usr/sbin/munin23:10:243
241575099391,35cyclictest2489664-21/usr/sbin/munin22:40:243
241575099391,35cyclictest2487620-21/usr/sbin/munin22:35:223
241575099391,35cyclictest2461972-21/usr/sbin/munin21:20:233
241575099391,35cyclictest2450163-21/usr/sbin/munin20:45:253
241575099391,35cyclictest2450163-21/usr/sbin/munin20:45:253
241575099391,35cyclictest2441150-21/usr/sbin/munin20:20:233
241575099391,34cyclictest2527110-21fschecks_count00:30:143
241575099391,33cyclictest2490866-21ntp_states22:45:213
241575099391,33cyclictest2464806-21df21:30:143
241575099391,33cyclictest2459364-21apt-get21:15:123
241575099391,33cyclictest2433580-21/usr/sbin/munin20:00:133
241575099391,32cyclictest2483471-21apt-get22:25:113
241575099381,34cyclictest2522934-21sshd00:15:423
241575099381,34cyclictest2520253-21diskstats00:10:163
241575099381,34cyclictest2518017-21cron00:03:003
241575099381,34cyclictest2518017-21cron00:03:003
241575099381,34cyclictest2478650-21/usr/sbin/munin22:10:163
241575099381,34cyclictest2473703-21/usr/sbin/munin21:55:203
241575099381,34cyclictest2473703-21/usr/sbin/munin21:55:203
241575099381,34cyclictest2467780-21cron21:39:003
241575099381,34cyclictest2452799-21/usr/sbin/munin20:55:163
241575099381,34cyclictest2444363-21/usr/sbin/munin20:30:193
241575099381,32cyclictest2502776-21df_inode23:20:153
241575099381,32cyclictest2420044-21df_inode19:20:153
241575099380,32cyclictest2474945-21apt-get22:00:133
241575099380,28cyclictest1592335-1kworker/3:2H+kblockd21:00:103
241575099380,28cyclictest1592335-1kworker/3:2H+kblockd21:00:103
2415746993823,12cyclictest311ktimers/223:10:112
2415746993817,4cyclictest131rcu_preempt21:50:232
2415746993727,6cyclictest0-21swapper/223:00:002
2415743993727,4cyclictest2487988-21kworker/u8:1+events_unbound22:36:411
241575099362,31cyclictest2456076-21diskstats21:05:153
241575099361,30cyclictest2506402-21/usr/sbin/munin23:30:163
241575099360,33cyclictest0-21swapper/323:00:123
241574699369,8cyclictest32-21ksoftirqd/200:20:002
2415746993618,5cyclictest131rcu_preempt23:15:222
2415743993626,5cyclictest0-21swapper/100:28:381
2415739993627,5cyclictest2444799-21unixbench_multi20:30:230
2415739993627,4cyclictest0-21swapper/023:12:080
2415739993622,9cyclictest2450199-21grep20:45:250
2415739993622,9cyclictest2450199-21grep20:45:250
241574699359,13cyclictest131rcu_preempt00:05:202
2415746993527,4cyclictest0-21swapper/221:46:532
2415746993526,6cyclictest0-21swapper/220:40:012
2415746993523,5cyclictest131rcu_preempt23:55:202
2415746993519,3cyclictest131rcu_preempt21:55:172
2415746993519,3cyclictest131rcu_preempt21:55:172
2415743993524,7cyclictest0-21swapper/123:00:001
2415739993523,9cyclictest121ktimers/022:55:120
2415750993416,14cyclictest0-21swapper/319:45:133
2415746993428,4cyclictest2454009-21cron21:00:002
2415746993427,5cyclictest2415301-21kworker/u8:0+events_unbound19:15:422
2415746993427,4cyclictest0-21swapper/222:45:002
2415746993425,5cyclictest301rcuc/220:15:012
2415746993420,4cyclictest131rcu_preempt20:55:002
2415746993418,3cyclictest131rcu_preempt21:30:232
2415743993427,4cyclictest0-21swapper/123:05:241
2415743993427,4cyclictest0-21swapper/120:40:011
2415743993426,5cyclictest0-21swapper/119:15:181
2415739993428,3cyclictest2423105-21latency_hist19:30:010
2415739993426,4cyclictest2446980-21munin-run20:40:010
2415739993425,5cyclictest2503691-21grep23:20:250
2415739993423,7cyclictest2462766-21debian-sa121:25:000
241573999341,5cyclictest2460374-21grep21:15:240
241574699338,15cyclictest32-21ksoftirqd/223:40:152
241574699335,15cyclictest32-21ksoftirqd/200:25:242
241574699333,15cyclictest32-21ksoftirqd/221:40:222
2415746993327,4cyclictest2426465-21kworker/u8:1+events_unbound19:44:422
2415746993327,4cyclictest2426465-21kworker/u8:1+events_unbound19:44:422
2415746993317,5cyclictest131rcu_preempt23:20:212
2415746993310,3cyclictest0-21swapper/223:10:002
2415743993326,5cyclictest2465856-21kworker/u8:1+events_unbound21:38:411
2415743993325,4cyclictest0-21swapper/120:20:011
2415739993325,4cyclictest0-21swapper/023:15:250
241574699326,15cyclictest32-21ksoftirqd/222:25:152
2415746993225,4cyclictest2455735-21munin-plugin-st21:05:012
2415746993225,4cyclictest2455735-21munin-plugin-st21:05:012
2415746993225,3cyclictest0-21swapper/200:30:142
2415746993224,4cyclictest0-21swapper/223:00:252
2415746993222,7cyclictest311ktimers/219:50:002
2415746993222,4cyclictest131rcu_preempt22:05:332
2415746993221,8cyclictest311ktimers/222:45:232
2415746993221,5cyclictest131rcu_preempt22:10:512
2415746993220,5cyclictest2413583-21kworker/u8:3+events_unbound19:14:542
2415746993220,5cyclictest2413583-21kworker/u8:3+events_unbound19:14:542
2415746993217,7cyclictest131rcu_preempt20:45:252
2415746993217,7cyclictest131rcu_preempt20:45:252
2415746993214,4cyclictest131rcu_preempt21:16:522
2415746993210,5cyclictest131rcu_preempt19:20:222
241574399323,5cyclictest131rcu_preempt00:20:141
2415743993225,5cyclictest2424737-21kworker/u8:0+events_unbound19:42:431
2415743993225,5cyclictest2424737-21kworker/u8:0+events_unbound19:42:431
2415743993225,4cyclictest0-21swapper/120:05:161
2415743993225,4cyclictest0-21swapper/119:25:161
2415743993222,7cyclictest0-21swapper/123:15:011
2415739993224,4cyclictest2464471-21kthreadd21:30:000
2415739993222,6cyclictest0-21swapper/022:55:000
2415739993220,8cyclictest2477712-21grep22:05:250
2415739993220,8cyclictest2416943-21grep19:10:230
2415739993220,8cyclictest2416943-21grep19:10:230
241573999321,29cyclictest0-21swapper/023:40:110
241574699319,5cyclictest131rcu_preempt22:00:212
241574699317,7cyclictest32-21ksoftirqd/221:25:182
241574699317,5cyclictest32-21ksoftirqd/200:35:212
2415746993120,4cyclictest131rcu_preempt19:25:172
2415746993117,3cyclictest131rcu_preempt21:05:182
2415746993115,9cyclictest131rcu_preempt20:15:152
2415746993110,5cyclictest131rcu_preempt20:20:222
2415743993117,11cyclictest2415730-21cyclictest21:20:111
241573999319,19cyclictest2512266-21timerandwakeup23:45:240
2415739993124,4cyclictest2426508-21munin-plugin-st19:40:010
2415739993124,3cyclictest0-21swapper/022:15:240
2415739993123,4cyclictest0-21swapper/000:13:330
2415739993122,6cyclictest2491759-21sh22:50:000
2415739993122,5cyclictest2528435-21munin-plugin-st00:35:000
2415739993121,6cyclictest2443143-21grep20:25:260
241574699309,12cyclictest131rcu_preempt21:10:232
241574699308,13cyclictest32-21ksoftirqd/220:05:162
241574699305,6cyclictest32-21ksoftirqd/221:20:202
2415746993020,7cyclictest311ktimers/223:50:222
2415746993020,7cyclictest301rcuc/220:32:562
2415746993020,7cyclictest2481102-21grep22:15:242
2415746993019,8cyclictest301rcuc/222:38:062
2415746993018,5cyclictest131rcu_preempt20:25:262
2415746993017,7cyclictest2478079-21kworker/u8:0+events_unbound23:28:512
2415746993013,9cyclictest131rcu_preempt19:30:222
2415746993013,3cyclictest131rcu_preempt22:20:182
2415746993010,6cyclictest2415730-21cyclictest19:37:172
2415746993010,5cyclictest131rcu_preempt20:00:142
2415743993022,4cyclictest0-21swapper/121:30:001
2415743993013,10cyclictest2469932-21df_inode21:45:131
2415739993023,4cyclictest2424368-21grep19:30:240
2415739993023,3cyclictest0-21swapper/022:35:140
2415739993022,5cyclictest2438437-21cron20:15:010
2415739993021,6cyclictest0-21swapper/020:00:000
2415739993020,6cyclictest0-21swapper/020:40:250
2415739993019,6cyclictest0-21swapper/000:20:170
241573999301,5cyclictest2428215-21cron19:45:010
241573999301,5cyclictest2428215-21cron19:45:010
241574699296,6cyclictest32-21ksoftirqd/220:40:022
2415746992920,6cyclictest0-21swapper/221:35:252
2415746992919,7cyclictest311ktimers/223:37:012
2415746992919,7cyclictest311ktimers/223:30:122
2415746992918,3cyclictest131rcu_preempt22:35:012
2415746992911,3cyclictest0-21swapper/200:00:022
2415746992911,3cyclictest0-21swapper/200:00:022
2415746992911,10cyclictest131rcu_preempt23:45:032
2415743992923,3cyclictest0-21swapper/120:10:181
2415743992922,5cyclictest2413583-21kworker/u8:3+events_unbound19:21:371
2415743992921,6cyclictest2514362-21kworker/u8:1+events_unbound23:55:171
2415746992811,4cyclictest131rcu_preempt19:55:112
2415746992810,4cyclictest131rcu_preempt00:20:182
2415743992822,4cyclictest2504059-21kworker/u8:1+events_unbound23:27:121
241574399281,5cyclictest131rcu_preempt00:00:121
241574399281,5cyclictest131rcu_preempt00:00:121
241574399281,3cyclictest131rcu_preempt22:05:121
241574399280,5cyclictest131rcu_preempt23:35:111
2415739992821,3cyclictest0-21swapper/021:35:250
2415739992818,6cyclictest2508792-21grep23:35:240
241573999281,5cyclictest2428669-21uniq19:45:150
2415743992722,3cyclictest2359521-21kworker/1:1-mm_percpu_wq21:41:371
2415743992721,4cyclictest2474378-21kworker/u8:2+events_unbound22:02:001
2415743992720,5cyclictest2432949-21kworker/u8:2+events_unbound20:01:471
2415743992720,4cyclictest0-21swapper/122:00:001
2415743992720,4cyclictest0-21swapper/122:00:001
2415743992720,4cyclictest0-21swapper/120:55:251
2415743992720,4cyclictest0-21swapper/120:50:161
241574399271,5cyclictest2417396-21latency_hist19:15:011
241574399271,5cyclictest2417396-21latency_hist19:15:011
2415743992711,6cyclictest2428202-21kworker/u8:2+events_unbound19:50:131
2415743992711,6cyclictest2428202-21kworker/u8:2+events_unbound19:50:131
2415743992711,12cyclictest221rcuc/121:05:321
2415739992722,3cyclictest13850irq/36-mmc120:05:000
2415739992720,4cyclictest2530149-21sendmail00:40:000
2415739992717,7cyclictest2519908-21munin-run00:10:000
2415739992717,6cyclictest0-21swapper/020:20:010
2415739992715,8cyclictest204-21dbus-daemon21:56:570
2415739992715,8cyclictest204-21dbus-daemon21:56:570
241574399264,5cyclictest2481934-21fschecks_count22:20:171
2415743992620,4cyclictest2478079-21kworker/u8:0+events_unbound22:27:361
2415743992620,4cyclictest2438344-21kworker/u8:1+events_unbound20:21:451
2415743992620,3cyclictest0-21swapper/119:50:001
2415743992619,5cyclictest2478079-21kworker/u8:0+events_unbound22:30:171
2415743992618,5cyclictest0-21swapper/123:20:251
2415743992618,5cyclictest0-21swapper/120:30:141
2415743992618,4cyclictest0-21swapper/119:35:251
2415743992614,4cyclictest131rcu_preempt23:55:011
241574399261,3cyclictest131rcu_preempt00:10:131
2415743992611,4cyclictest131rcu_preempt00:35:131
241573999262,5cyclictest2486238-21grep22:30:280
2415739992620,3cyclictest0-21swapper/021:10:000
2415739992619,3cyclictest0-21swapper/021:35:000
2415739992618,5cyclictest0-21swapper/020:50:160
2415739992618,5cyclictest0-21swapper/000:20:000
2415739992618,4cyclictest0-21swapper/023:25:260
241573999261,5cyclictest2495625-21apt-get23:00:120
2415739992614,9cyclictest2525888-21cut00:25:230
241574399258,9cyclictest2479419-21users22:10:241
241574399254,3cyclictest131rcu_preempt20:40:131
2415743992519,4cyclictest2528041-21kworker/u8:1+events_unbound00:34:441
2415743992519,4cyclictest2458748-21kworker/u8:1+events_unbound21:11:571
2415743992517,5cyclictest0-21swapper/123:45:151
2415743992514,8cyclictest24-21ksoftirqd/122:45:181
241574399251,11cyclictest0-21swapper/100:18:441
2415743992510,6cyclictest2430281-21kworker/u8:1+events_unbound19:59:591
2415739992519,3cyclictest0-21swapper/022:05:000
2415739992519,3cyclictest0-21swapper/021:40:140
2415739992518,3cyclictest0-21swapper/022:20:270
2415739992517,4cyclictest13550irq/42-30be0000.ethernet22:25:230
2415739992516,6cyclictest0-21swapper/023:30:230
2415739992514,7cyclictest2431145-21grep19:50:240
2415739992514,7cyclictest2431145-21grep19:50:240
241574399248,9cyclictest0-21swapper/123:30:221
2415743992418,4cyclictest2519010-21kworker/u8:3+events_unbound00:08:021
2415743992418,4cyclictest2464146-21kworker/u8:2+events_unbound21:30:131
2415743992410,3cyclictest131rcu_preempt20:45:191
2415743992410,3cyclictest131rcu_preempt20:45:191
241573999241,9cyclictest2436753-21munin-run20:10:010
2415739992418,3cyclictest0-21swapper/019:20:230
2415739992416,5cyclictest2427354-1kworker/0:1H+mmc_complete21:55:000
241573999241,19cyclictest0-21swapper/023:54:420
241573999241,19cyclictest0-21swapper/022:11:430
2415739992316,4cyclictest0-21swapper/021:10:210
2415739992316,3cyclictest0-21swapper/023:05:240
241573999231,20cyclictest0-21swapper/020:20:120
241573999231,18cyclictest2516308-21basename00:00:000
2415739992311,8cyclictest2470490-21cut21:45:210
241573999231,18cyclictest0-21swapper/000:00:530
241573999231,18cyclictest0-21swapper/000:00:530
241574399227,8cyclictest131rcu_preempt23:17:101
241574399226,7cyclictest131rcu_preempt22:52:521
241574399221,4cyclictest0-21swapper/123:00:221
241573999222,15cyclictest2455313-21unixbench_singl21:00:240
241573999222,15cyclictest2455313-21unixbench_singl21:00:240
241573999220,18cyclictest0-21swapper/022:42:370
2415743992113,5cyclictest0-21swapper/121:00:141
2415743992113,5cyclictest0-21swapper/121:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional