You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 06:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue Dec 16, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
561396993731,4cyclictest857242-21cat23:55:010
561404993327,4cyclictest252-21systemd-journal20:20:002
561396993226,4cyclictest772833-21latency_hist22:35:000
561404992824,3cyclictest726582-21kworker/u8:1+flush-259:021:55:192
561400992822,4cyclictest545134-21kworker/u8:3+flush-259:019:40:001
561396992721,4cyclictest566480-21latency_hist19:15:010
561396992720,5cyclictest850642-21ntpq23:45:210
561400992619,5cyclictest894184-21latency_hist00:30:011
561396992619,5cyclictest819247-21ntpq23:15:250
561396992619,5cyclictest819247-21ntpq23:15:250
561400992521,3cyclictest772859-21kworker/u8:1+flush-259:022:45:001
561400992519,3cyclictest659142-21kworker/u8:0+flush-259:020:50:171
561396992519,4cyclictest857757-21apt-get23:55:130
561409992417,4cyclictest664797-21latency_hist20:50:003
561409992417,4cyclictest664797-21latency_hist20:50:003
561400992418,4cyclictest737075-21/usr/sbin/munin22:00:151
561409992317,4cyclictest648963-21latency_hist20:35:003
561404992318,4cyclictest857632-21gpgv23:55:012
561404992317,4cyclictest623179-21latency_hist20:10:002
561400992318,3cyclictest798831-21kworker/u8:2+events_unbound23:10:011
561400992316,5cyclictest545134-21kworker/u8:3+flush-259:019:15:181
561396992319,3cyclictest700585-21kworker/u8:1+events_unbound21:30:010
56140499220,5cyclictest725625-21/usr/sbin/munin21:45:212
561396992218,3cyclictest664825-21kworker/u8:2+flush-259:020:55:220
561404992119,1cyclictest894175-21cron00:30:012
561404992117,3cyclictest778090-21gpgv22:39:592
561404992115,4cyclictest565662-21ntpq19:10:232
561396992118,2cyclictest633557-21kworker/u8:3+events_unbound21:10:260
561409992016,3cyclictest252-21systemd-journal21:07:173
561404992017,2cyclictest836008-21idleruntime-cro23:35:002
561404992015,3cyclictest607942-21kworker/u8:1+flush-259:020:25:182
561396992016,3cyclictest757319-21kworker/u8:3+flush-259:023:03:170
561409991915,3cyclictest0-21swapper/319:15:003
561409991914,3cyclictest809365-21latency_hist23:10:013
561409991914,3cyclictest757417-21latency_hist22:20:023
561409991913,4cyclictest716282-21latency_hist21:40:013
561404991916,1cyclictest0-21swapper/221:15:002
561404991915,3cyclictest762797-21apt-get22:25:122
561404991913,4cyclictest684568-21ntpq21:05:212
561404991913,4cyclictest680239-21latency_hist21:05:012
561400991916,2cyclictest863811-21irqcore00:00:191
561400991914,4cyclictest757319-21kworker/u8:3+flush-259:022:20:001
561400991913,4cyclictest761615-21ntpq22:20:201
561400991912,5cyclictest252-21systemd-journal21:30:011
561396991916,2cyclictest556443-21kworker/u8:2+flush-259:019:50:000
561396991915,3cyclictest628025-21kworker/u8:2+flush-259:020:39:560
561396991912,5cyclictest586288-21ntpq19:30:220
561409991815,2cyclictest862688-21cron00:00:003
561409991811,5cyclictest904542-21latency_hist00:40:003
561404991816,1cyclictest793905-21sh22:55:002
56140499180,3cyclictest721161-21chrt21:42:162
56140499180,16cyclictest659237-21latency_hist20:45:002
561400991813,3cyclictest794161-21apt-get22:55:131
561396991815,2cyclictest857099-21kworker/u8:2+flush-259:000:05:160
561396991815,2cyclictest767684-21idleruntime-cro22:29:590
561396991814,3cyclictest757319-21kworker/u8:3+flush-259:022:59:590
561396991814,3cyclictest695364-21kworker/u8:2+flush-259:021:45:210
561396991814,3cyclictest60350irq/132-enp0s29f1:rx-019:50:200
561396991813,3cyclictest633557-21kworker/u8:3+flush-259:021:35:210
561396991812,4cyclictest809598-21cat23:10:010
561409991715,1cyclictest0-21swapper/322:39:593
561409991714,2cyclictest0-21swapper/322:50:003
561409991712,4cyclictest252-21systemd-journal00:15:013
561409991712,3cyclictest659265-21gpgv20:45:013
56140499179,7cyclictest711163-21munin-plugin-st21:35:012
56140499172,11cyclictest884121-21apt-get00:20:142
561404991711,5cyclictest0-21swapper/220:25:002
56140499171,14cyclictest820141-21idleruntime-cro23:20:002
56140499171,14cyclictest820141-21idleruntime-cro23:19:592
561404991711,4cyclictest804227-21latency_hist23:05:002
561404991710,5cyclictest814143-21ntpq23:10:222
561400991713,3cyclictest851519-21apt-get23:50:011
561396991714,2cyclictest659142-21kworker/u8:0+events_unbound20:45:210
561396991714,2cyclictest659142-21kworker/u8:0+events_unbound20:45:210
561396991714,2cyclictest628025-21kworker/u8:2+events_unbound20:40:230
561396991713,3cyclictest695364-21kworker/u8:2+events_unbound21:34:570
561396991713,3cyclictest545134-21kworker/u8:3+flush-259:019:40:210
561396991712,3cyclictest659142-21kworker/u8:0+flush-259:021:20:190
561409991610,4cyclictest868340-21latency_hist00:05:013
56140999160,9cyclictest582427-21diskstats19:30:153
56140499169,6cyclictest739403-21tr22:00:192
56140499164,10cyclictest628357-21idleruntime-cro20:15:002
56140499163,10cyclictest727160-21/usr/sbin/munin21:50:162
56140499161,6cyclictest787923-21ntpq22:45:222
56140499161,13cyclictest670658-21ps20:55:172
56140499161,12cyclictest669995-21basename20:55:002
56140499161,11cyclictest649225-21apt-get20:35:132
56140499160,1cyclictest5766382sleep219:23:202
56140499160,14cyclictest878683-21true00:15:012
56140499160,14cyclictest825286-21idleruntime-cro23:24:592
56140099169,5cyclictest711389-21/usr/sbin/munin21:35:141
561400991613,2cyclictest869629-21kthreadcore00:05:181
561400991613,2cyclictest0-21swapper/123:45:001
561400991612,3cyclictest747353-21/usr/sbin/munin22:10:171
561400991612,2cyclictest0-21swapper/119:25:001
561400991611,3cyclictest726582-21kworker/u8:1+flush-259:022:10:001
561400991610,4cyclictest643815-21latency_hist20:30:001
561396991613,2cyclictest633557-21kworker/u8:3+flush-259:021:40:270
561396991613,2cyclictest60350irq/132-enp0s29f1:rx-023:29:550
561396991613,2cyclictest60350irq/132-enp0s29f1:rx-022:43:240
561396991613,2cyclictest60350irq/132-enp0s29f1:rx-019:20:110
561396991613,2cyclictest60350irq/132-enp0s29f1:rx-000:15:250
561396991612,3cyclictest809396-21kworker/u8:4+events_unbound23:20:230
561396991612,3cyclictest772859-21kworker/u8:1+flush-259:022:45:200
561396991612,3cyclictest752492-21/usr/sbin/munin22:15:150
561396991612,3cyclictest628025-21kworker/u8:2+events_unbound20:30:220
561396991612,3cyclictest597438-21kworker/u8:0+events_unbound20:05:150
561396991611,4cyclictest659142-21kworker/u8:0+events_unbound21:19:560
561409991512,2cyclictest706002-21idleruntime-cro21:30:003
561409991512,2cyclictest690552-21cron21:15:003
561409991512,1cyclictest0-21swapper/323:35:003
561409991511,3cyclictest252-21systemd-journal22:20:553
561409991511,2cyclictest0-21swapper/323:20:003
561409991511,2cyclictest0-21swapper/323:19:593
56140999150,6cyclictest790690-21cut22:50:173
56140499159,5cyclictest857069-21apt-get23:55:002
56140499157,7cyclictest0-21swapper/222:20:192
56140499152,9cyclictest566751-21apt-get19:15:142
56140499151,1cyclictest617103-21ntpq20:00:212
56140499151,12cyclictest783576-21idleruntime-cro22:45:002
56140499151,12cyclictest591414-21ntpq19:35:222
56140499151,12cyclictest252-21systemd-journal23:45:002
56140499151,12cyclictest252-21systemd-journal22:15:012
561404991510,4cyclictest611992-21ntpq19:55:222
56140499150,6cyclictest756547-21ntpq22:15:222
56140499150,5cyclictest867870-21unixbench_multi00:00:282
56140499150,5cyclictest742611-21/usr/sbin/munin22:05:162
56140499150,4cyclictest851741-21wc23:50:012
56140499150,4cyclictest706006-21awk21:30:002
56140499150,3cyclictest904550-21turbostat.cron00:40:012
56140499150,3cyclictest825284-21cron23:25:002
56140499150,2cyclictest873550-21sh00:10:002
56140499150,2cyclictest799033-21awk23:00:002
56140499150,2cyclictest664813-21idleruntime-cro20:50:012
56140499150,2cyclictest664813-21idleruntime-cro20:50:002
56140499150,13cyclictest808897-21grep23:05:272
56140499150,12cyclictest644167-21tr20:30:142
56140499150,12cyclictest592983-21grep19:40:162
56140099159,5cyclictest606795-21ntpq19:50:231
56140099159,4cyclictest664825-21kworker/u8:2+flush-259:020:55:171
56140099159,4cyclictest618258-21/usr/sbin/munin20:05:151
561400991513,1cyclictest0-21swapper/100:20:001
561400991512,1cyclictest0-21swapper/123:55:001
561400991511,3cyclictest731984-21apt-get21:55:141
561396991513,2cyclictest60350irq/132-enp0s29f1:rx-020:22:200
561396991513,1cyclictest60350irq/132-enp0s29f1:rx-023:35:160
561396991513,1cyclictest60350irq/132-enp0s29f1:rx-023:34:220
561396991513,1cyclictest60350irq/132-enp0s29f1:rx-021:50:170
561396991513,1cyclictest60350irq/132-enp0s29f1:rx-020:01:470
561396991513,1cyclictest60350irq/132-enp0s29f1:rx-000:20:170
561396991512,2cyclictest804243-21kworker/u8:1+flush-259:000:10:550
561396991512,2cyclictest607942-21kworker/u8:1+events_unbound20:15:230
561396991512,2cyclictest60350irq/132-enp0s29f1:rx-022:50:010
561396991512,2cyclictest60350irq/132-enp0s29f1:rx-021:05:160
561396991512,2cyclictest60350irq/132-enp0s29f1:rx-020:12:460
561396991512,2cyclictest60350irq/132-enp0s29f1:rx-000:33:010
561396991512,2cyclictest60350irq/132-enp0s29f1:rx-000:00:160
56140999143,7cyclictest0-21swapper/322:00:203
561409991412,1cyclictest211-21jbd2/nvme0n1p2-821:25:013
561409991411,2cyclictest0-21swapper/323:40:003
56140999140,4cyclictest725740-21ld21:45:233
56140999140,4cyclictest596524-21ld19:40:213
56140499147,6cyclictest0-21swapper/221:20:172
56140499141,5cyclictest879335-21/usr/sbin/munin00:15:182
56140499141,2cyclictest586316-21ntpq19:30:232
56140499141,12cyclictest0-21swapper/219:10:012
56140499140,4cyclictest691162-21cat21:15:192
56140499140,4cyclictest606998-21/usr/sbin/munin19:50:262
56140499140,4cyclictest598096-21irqcore19:45:162
56140499140,2cyclictest771869-21grep22:30:202
56140499140,2cyclictest581113-21ntpq19:25:222
56140499140,12cyclictest711526-21cut21:35:132
56140099149,4cyclictest659239-21apt-get20:45:011
56140099149,4cyclictest659239-21apt-get20:45:001
561400991411,2cyclictest613550-21irqcore20:00:171
561400991411,1cyclictest612868-21idleruntime-cro19:59:591
561400991410,2cyclictest0-21swapper/123:35:231
561396991412,2cyclictest60350irq/132-enp0s29f1:rx-019:18:450
561396991412,1cyclictest60350irq/132-enp0s29f1:rx-021:55:150
561396991411,2cyclictest60350irq/132-enp0s29f1:rx-022:20:170
561396991411,2cyclictest60350irq/132-enp0s29f1:rx-020:29:330
561396991411,2cyclictest60350irq/132-enp0s29f1:rx-000:29:560
56140999137,4cyclictest873521-21sshd00:09:543
561409991310,2cyclictest857067-21cron23:55:013
561409991310,2cyclictest762879-21cpuspeed_turbos22:25:143
561409991310,2cyclictest664825-21kworker/u8:2+flush-259:021:00:003
561409991310,2cyclictest633097-21unixbench_singl20:15:253
561409991310,2cyclictest566818-21cpuspeed_turbos19:15:143
561409991310,2cyclictest0-21swapper/319:54:593
56140999130,5cyclictest628207-21taskset20:13:123
56140999130,3cyclictest601502-21memory19:45:203
56140099139,3cyclictest762864-21cpuspeed_turbos22:25:141
56140099137,4cyclictest695364-21kworker/u8:2+flush-259:021:50:001
561400991311,1cyclictest0-21swapper/120:10:261
561400991310,2cyclictest563896-21kthreadcore19:10:191
561400991310,2cyclictest0-21swapper/100:00:011
56139699139,3cyclictest566507-21kworker/u8:0+flush-259:019:25:210
56139699137,4cyclictest509-21rs:main0
561396991311,1cyclictest60350irq/132-enp0s29f1:rx-023:10:190
561396991310,2cyclictest804243-21kworker/u8:1+flush-259:023:40:350
561396991310,2cyclictest60350irq/132-enp0s29f1:rx-022:05:150
561396991310,2cyclictest60350irq/132-enp0s29f1:rx-020:52:060
561396991310,2cyclictest60350irq/132-enp0s29f1:rx-000:35:170
56140999129,2cyclictest735828-21needreboot21:55:203
56140999129,2cyclictest695273-21unixbench_singl21:15:283
56140999129,2cyclictest639055-21cpuspeed_turbos20:25:163
56140999129,2cyclictest622776-21unixbench_singl20:05:293
56140999128,3cyclictest825021-21sshd23:21:553
561409991210,1cyclictest717548-21kernelversion21:40:183
56140999120,4cyclictest612526-21sshd19:55:553
56140099129,2cyclictest690143-21unixbench_singl21:10:271
56140099129,2cyclictest597816-21cpuspeed_turbos19:45:131
56140099128,3cyclictest900662-21kthreadcore00:35:181
56140099128,3cyclictest894509-21cpuspeed_turbos00:30:111
56140099126,4cyclictest783823-21/usr/sbin/munin22:45:201
56140099126,4cyclictest628025-21kworker/u8:2+flush-259:020:40:171
56140099120,10cyclictest772840-21awk22:34:591
56139699129,2cyclictest772859-21kworker/u8:1+flush-259:022:35:260
56139699129,2cyclictest60350irq/132-enp0s29f1:rx-022:00:150
56140999119,1cyclictest890944-21kthreadcore00:25:193
56140999118,2cyclictest826408-21irqcore23:25:173
56140999118,2cyclictest778787-21cpuspeed_turbos22:40:143
56140999118,2cyclictest577469-21irqcore19:25:173
56140999116,4cyclictest731264-21grep21:50:253
56140999116,3cyclictest616945-21memory20:00:193
56140999110,4cyclictest798481-21sed22:55:283
56140099118,2cyclictest798831-21kworker/u8:2+flush-259:023:15:201
56140099118,2cyclictest798831-21kworker/u8:2+flush-259:023:15:201
56140099118,2cyclictest628890-21fschecks_time20:15:151
56140999106,3cyclictest591837-21grep19:35:253
56140999105,3cyclictest669104-21gcc20:50:223
56140999100,2cyclictest751439-21ntp_states22:10:243
56140099107,2cyclictest582332-21cpuspeed_turbos19:30:141
56140099107,2cyclictest577124-21cpuspeed_turbos19:25:151
56140099106,3cyclictest675545-21cut21:00:161
56140099104,4cyclictest1-21systemd22:40:001
5614009995,3cyclictest684568-21ntpq21:05:211
5614009991,6cyclictest700844-21awk21:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional