You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 09:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Apr 03, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
628755995547,5cyclictest653350-21cron20:20:001
628757995241,7cyclictest673617-21grep21:15:272
628757995233,16cyclictest742696-21cron00:40:002
628761995137,11cyclictest391ktimers/322:15:123
628757995135,12cyclictest311ktimers/221:05:152
628757995135,12cyclictest311ktimers/221:05:142
628757994836,9cyclictest311ktimers/219:55:112
628755994841,4cyclictest735893-21sendmail00:20:001
628755994733,10cyclictest231ktimers/122:45:141
628757994523,18cyclictest0-21swapper/220:20:242
628757994424,16cyclictest637802-21auditctl19:35:002
62875799441,10cyclictest717590-21/usr/sbin/munin23:25:212
628750994330,10cyclictest708359-21cron23:00:010
628761994229,9cyclictest742697-21sendmail00:40:003
62875799421,9cyclictest707449-21/usr/sbin/munin22:55:212
62875799421,9cyclictest679532-21/usr/sbin/munin21:35:142
62875799421,8cyclictest703502-21/usr/sbin/munin22:45:142
62875799421,8cyclictest671215-21/usr/sbin/munin21:10:192
62875799421,8cyclictest671215-21/usr/sbin/munin21:10:192
62875799421,10cyclictest639159-21cron19:36:002
628755994235,4cyclictest742715-21sendmail00:40:001
628755994230,8cyclictest231ktimers/119:15:131
62875799413,5cyclictest684520-21apt-get21:50:132
628757994131,7cyclictest311ktimers/222:00:112
628757994131,6cyclictest642485-21grep19:45:242
62875799411,8cyclictest720128-21/usr/sbin/munin23:30:282
62875799411,7cyclictest652559-21/usr/sbin/munin20:15:232
62875799411,11cyclictest731334-21irqstats00:05:172
628755994133,5cyclictest715206-21sendmail23:20:001
628761994029,7cyclictest666753-21grep20:55:253
62875799406,8cyclictest682102-21grep21:40:262
62875799403,6cyclictest656386-21users20:25:262
62875799403,5cyclictest711087-21/usr/sbin/munin23:05:242
62875799401,9cyclictest701928-21/usr/sbin/munin22:40:172
62875799401,9cyclictest678267-21ntp_states21:30:222
62875799401,9cyclictest629-21snmpd22:20:442
62875799401,7cyclictest726891-21/usr/sbin/munin23:50:242
62875799401,7cyclictest686886-21/usr/sbin/munin21:55:222
62875799401,7cyclictest682937-21/usr/sbin/munin21:45:142
628755994032,4cyclictest0-21swapper/121:40:011
628755994030,6cyclictest737982-21cut00:25:161
628755994028,9cyclictest653948-21kworker/u8:3+events_unbound20:30:111
628750994032,5cyclictest637364-21unixbench_singl19:30:240
628750994031,5cyclictest668485-21sed21:00:230
62875799392,9cyclictest643605-21/usr/sbin/munin19:50:202
62875799392,6cyclictest738505-21/usr/sbin/munin00:25:242
62875799392,4cyclictest676109-21df_inode21:25:142
62875799392,10cyclictest722469-21apt-get23:40:132
62875799391,9cyclictest727754-21/usr/sbin/munin23:55:152
62875799391,9cyclictest693147-21/usr/sbin/munin22:15:142
62875799391,9cyclictest661365-21/usr/sbin/munin20:40:232
62875799391,9cyclictest651028-21unixbench_singl20:10:272
62875799391,9cyclictest633378-21meminfo19:20:192
62875799391,4cyclictest700075-21/usr/sbin/munin22:35:242
62875799391,11cyclictest714166-21memory23:15:202
62875799391,11cyclictest705305-21df_inode22:50:112
628755993933,4cyclictest687693-21latency_hist22:00:001
628755993931,5cyclictest660399-21munin-plugin-st20:40:001
628761993830,5cyclictest730335-21grep00:00:263
62875799381,9cyclictest690088-21meminfo22:05:192
62875799381,9cyclictest634984-21/usr/sbin/munin19:25:182
62875799381,8cyclictest674387-21/usr/sbin/munin21:20:142
62875799381,6cyclictest631036-21diskstats19:15:042
62875799381,11cyclictest662455-21df_inode20:45:132
62875799381,11cyclictest648726-21memory20:05:202
62875799381,10cyclictest724287-21df_inode23:45:152
62875799381,10cyclictest712562-21tr23:10:212
628755993827,7cyclictest0-21swapper/120:10:251
628761993731,4cyclictest632343-21kworker/u8:1+events_unbound19:27:463
62875799373,5cyclictest734846-21memory00:15:192
62875799371,9cyclictest739661-21fschecks_count00:30:182
62875799371,9cyclictest696916-21memory22:25:212
62875799371,8cyclictest699317-21seq22:31:222
62875799371,8cyclictest53-21kcompactd020:53:202
62875799371,7cyclictest729293-21/usr/sbin/munin00:00:162
62875799371,7cyclictest715435-21/usr/sbin/munin23:20:152
62875799371,10cyclictest733260-21/usr/sbin/munin00:10:212
62875799371,10cyclictest691792-21memory22:10:212
628755993730,5cyclictest721847-21kworker/u8:3+events_unbound23:43:121
628755993727,6cyclictest0-21swapper/122:05:251
628761993629,5cyclictest636040-21kworker/u8:2+events_unbound19:42:483
628761993625,7cyclictest0-21swapper/320:40:253
628757993623,9cyclictest668929-21sh21:05:002
62875799361,9cyclictest737056-21/usr/sbin/munin00:20:252
62875799361,8cyclictest720870-21diskstats23:35:152
62875799361,8cyclictest629035-21forks19:10:152
62875799361,7cyclictest708725-21/usr/sbin/munin23:00:142
62875799361,7cyclictest640862-21seq19:40:472
62875799360,9cyclictest657323-21cstates20:30:132
628755993628,4cyclictest708368-21latency_hist23:00:001
628755993626,7cyclictest0-21swapper/122:10:261
628755993625,7cyclictest0-21swapper/121:20:001
628761993528,4cyclictest729050-21cron00:00:003
62875799351,8cyclictest659259-21irqstats20:35:172
62875799351,7cyclictest646570-21/usr/sbin/munin20:00:142
628755993528,5cyclictest670269-21kworker/u8:1+events_unbound21:06:421
628755993528,5cyclictest670269-21kworker/u8:1+events_unbound21:06:411
628755993527,4cyclictest0-21swapper/122:53:091
628755993526,6cyclictest645876-21tail19:55:261
628750993528,4cyclictest655075-21cron20:25:000
628750993527,4cyclictest725639-21apt23:50:000
628761993426,4cyclictest0-21swapper/322:40:013
628761993424,7cyclictest0-21swapper/323:40:003
628761993423,7cyclictest670187-21grep21:05:263
628761993423,7cyclictest670187-21grep21:05:263
628755993424,6cyclictest0-21swapper/122:25:011
628755993423,7cyclictest666786-21grep20:55:241
628750993427,4cyclictest680878-21munin-run21:40:020
628750993425,6cyclictest0-21swapper/023:30:100
628761993327,4cyclictest733849-21kworker/u8:0+events_unbound00:13:153
628761993327,4cyclictest702963-21kworker/u8:2+events_unbound00:21:413
628761993326,3cyclictest0-21swapper/321:10:283
628761993326,3cyclictest0-21swapper/321:10:273
628761993325,4cyclictest0-21swapper/321:05:003
628761993324,4cyclictest0-21swapper/319:15:263
628761993323,6cyclictest702834-21grep22:40:263
628761993323,6cyclictest0-21swapper/319:40:003
628755993323,6cyclictest0-21swapper/123:30:001
628755993322,7cyclictest640786-21grep19:40:261
628761993222,6cyclictest0-21swapper/320:15:243
628761993220,8cyclictest696229-21apt22:25:003
628755993220,9cyclictest231ktimers/120:40:131
628750993224,4cyclictest0-21swapper/023:40:000
628761993124,4cyclictest0-21swapper/320:10:163
628755993121,6cyclictest0-21swapper/122:25:251
628750993124,4cyclictest667222-21sendmail21:00:000
628750993120,7cyclictest732468-21idleruntime-cro00:10:000
628761993023,5cyclictest391ktimers/319:30:123
628761993022,5cyclictest0-21swapper/322:05:263
628761993022,4cyclictest0-21swapper/322:25:163
628761993021,4cyclictest0-21swapper/320:05:193
62875599303,6cyclictest131rcu_preempt22:15:131
628755993024,3cyclictest0-21swapper/121:20:161
628755993023,5cyclictest702963-21kworker/u8:2+events_unbound23:33:111
628755993020,7cyclictest0-21swapper/100:20:251
62875099308,3cyclictest6298752cat19:10:240
628761992921,4cyclictest0-21swapper/323:45:283
628761992919,7cyclictest0-21swapper/322:45:123
628761992919,5cyclictest687717-21sendmail22:00:003
628755992920,6cyclictest0-21swapper/122:45:001
628755992920,5cyclictest0-21swapper/100:10:251
628750992922,4cyclictest701567-21sendmail22:40:010
628750992921,3cyclictest0-21swapper/019:40:000
628750992920,6cyclictest688124-21grep22:00:160
628750992918,7cyclictest0-21swapper/020:55:010
62876199284,4cyclictest131rcu_preempt23:00:103
628761992822,4cyclictest0-21swapper/323:50:123
628761992820,4cyclictest0-21swapper/322:35:013
628755992820,4cyclictest0-21swapper/100:33:161
628755992819,6cyclictest0-21swapper/123:35:261
628755992818,7cyclictest231ktimers/100:05:131
628755992812,5cyclictest625013-21kworker/u8:0+events_unbound20:27:561
628750992819,6cyclictest0-21swapper/023:55:000
628750992819,5cyclictest694543-21sendmail-msp22:20:000
628750992818,6cyclictest13550irq/42-30be0000.ethernet22:35:010
628761992721,4cyclictest675500-21kworker/u8:1+events_unbound23:30:403
628761992720,5cyclictest663500-21kworker/u8:0+events_unbound20:48:413
628761992720,5cyclictest653948-21kworker/u8:3+events_unbound20:20:413
628761992718,5cyclictest0-21swapper/322:50:123
628761992717,6cyclictest0-21swapper/323:15:003
628761992714,9cyclictest711317-21grep23:05:253
628761992714,9cyclictest656375-21grep20:25:263
628755992720,4cyclictest0-21swapper/119:50:161
628755992719,5cyclictest0-21swapper/119:45:241
628755992719,5cyclictest0-21swapper/100:00:261
628755992719,4cyclictest0-21swapper/121:50:271
628755992718,6cyclictest0-21swapper/120:55:001
628755992718,6cyclictest0-21swapper/119:30:251
628755992716,7cyclictest0-21swapper/121:48:031
628750992714,8cyclictest0-21swapper/020:35:140
628761992620,4cyclictest721847-21kworker/u8:3+events_unbound23:42:383
628761992620,4cyclictest672239-21kworker/u8:0+events_unbound21:53:413
628761992619,5cyclictest656637-21kworker/u8:2+events_unbound21:20:253
628761992618,5cyclictest0-21swapper/300:15:163
628761992618,3cyclictest0-21swapper/319:10:243
628755992619,4cyclictest0-21swapper/123:10:171
628755992617,6cyclictest0-21swapper/123:55:141
628755992617,6cyclictest0-21swapper/122:35:271
628750992620,4cyclictest0-21swapper/023:55:110
628750992619,3cyclictest0-21swapper/020:30:260
628750992619,3cyclictest0-21swapper/020:15:010
628750992617,6cyclictest648040-21sh20:05:020
628750992617,6cyclictest0-21swapper/021:20:000
628750992617,4cyclictest0-21swapper/023:25:260
628750992616,5cyclictest649321-21grep20:05:250
628761992519,4cyclictest702963-21kworker/u8:2-events_freezable_power_23:15:153
628761992518,4cyclictest0-21swapper/320:30:223
628761992517,4cyclictest646342-21sendmail20:00:013
628755992519,3cyclictest0-21swapper/120:05:241
628755992518,4cyclictest0-21swapper/119:25:151
628755992517,4cyclictest0-21swapper/121:10:271
628755992517,4cyclictest0-21swapper/121:10:271
628755992515,7cyclictest716127-21fgrep23:20:241
628750992519,3cyclictest0-21swapper/019:15:040
628750992518,4cyclictest704045-21ntpq22:45:210
628750992518,4cyclictest672239-21kworker/u8:0-writeback22:50:120
628750992518,4cyclictest13850irq/36-mmc120:00:010
628750992518,4cyclictest0-21swapper/022:40:260
628750992518,4cyclictest0-21swapper/021:25:270
628750992518,4cyclictest0-21swapper/000:30:280
628750992517,6cyclictest0-21swapper/000:20:150
628750992516,4cyclictest0-21swapper/023:10:150
628750992513,8cyclictest204-21dbus-daemon19:47:480
62875099251,21cyclictest0-21swapper/000:30:000
62875099251,20cyclictest711316-21grep23:05:250
628750992511,10cyclictest121ktimers/022:20:140
62875099250,21cyclictest0-21swapper/020:16:120
628761992418,4cyclictest702963-21kworker/u8:2+events_unbound22:56:423
628761992418,3cyclictest0-21swapper/321:47:403
628761992417,3cyclictest0-21swapper/323:30:003
628761992416,3cyclictest0-21swapper/321:15:173
628761992415,6cyclictest0-21swapper/300:10:003
628761992410,4cyclictest702963-21kworker/u8:2+events_unbound23:24:563
62875599242,16cyclictest0-21swapper/119:20:101
628755992418,4cyclictest705639-21kworker/u8:3+events_unbound23:00:231
628755992416,5cyclictest654159-21ntpq20:20:211
628755992416,5cyclictest0-21swapper/121:30:151
628755992413,7cyclictest699831-21sed22:35:011
628750992418,3cyclictest0-21swapper/021:10:280
628750992418,3cyclictest0-21swapper/021:10:270
628750992418,3cyclictest0-21swapper/019:50:250
628750992417,4cyclictest0-21swapper/021:30:260
628750992417,4cyclictest0-21swapper/020:40:250
628750992415,6cyclictest0-21swapper/023:20:010
628750992415,6cyclictest0-21swapper/020:30:010
62875099241,19cyclictest0-21swapper/000:17:530
628761992316,5cyclictest391ktimers/319:52:363
628761992315,5cyclictest636040-21kworker/u8:2+events_unbound20:02:503
628761992313,6cyclictest0-21swapper/300:30:283
628755992317,3cyclictest0-21swapper/121:40:241
628750992318,3cyclictest0-21swapper/023:20:250
628750992317,3cyclictest0-21swapper/019:20:230
628750992316,4cyclictest733699-21grep00:10:260
628750992316,4cyclictest0-21swapper/021:50:270
628750992316,3cyclictest0-21swapper/019:41:560
62875099231,20cyclictest0-21swapper/021:06:280
62875099231,20cyclictest0-21swapper/021:06:280
62875099231,18cyclictest0-21swapper/022:26:580
62875099230,19cyclictest0-21swapper/023:41:480
62875099230,19cyclictest0-21swapper/023:03:520
62875099230,19cyclictest0-21swapper/021:57:070
62875099230,19cyclictest0-21swapper/021:45:310
62875099230,19cyclictest0-21swapper/021:22:150
628761992215,5cyclictest687307-21kworker/u8:2+events_unbound22:10:193
628761992215,5cyclictest653948-21kworker/u8:3+events_unbound20:51:413
628761992215,3cyclictest0-21swapper/321:25:213
628761992214,6cyclictest682787-21sed21:45:023
628761992212,7cyclictest679777-21grep21:35:193
62876199221,11cyclictest0-21swapper/321:31:013
62876199220,12cyclictest0-21swapper/322:00:263
628755992214,5cyclictest0-21swapper/123:05:151
628755992214,5cyclictest0-21swapper/121:25:271
628755992212,7cyclictest0-21swapper/120:00:171
62875099222,18cyclictest0-21swapper/022:11:170
628750992216,3cyclictest13550irq/42-30be0000.ethernet00:00:200
628750992215,4cyclictest0-21swapper/020:45:240
62875099220,18cyclictest0-21swapper/021:42:470
628761992115,3cyclictest0-21swapper/320:35:163
62875599211,7cyclictest688940-21tail22:00:251
628755992114,3cyclictest0-21swapper/119:35:131
628755992013,5cyclictest231ktimers/119:10:171
628755992013,4cyclictest0-21swapper/120:45:151
628755992012,5cyclictest0-21swapper/123:50:211
62875099200,18cyclictest0-21swapper/022:07:470
628761991913,4cyclictest625013-21kworker/u8:0+events_unbound19:22:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional