You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 10:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sun Sep 13, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275392241166,23sleep10-21swapper/119:09:111
272842237164,24sleep20-21swapper/219:05:522
274002203166,25sleep30-21swapper/319:07:223
273542202166,25sleep00-21swapper/019:06:480
2996821410,4sleep32769999cyclictest22:52:053
1026821380,5sleep02767999cyclictest19:40:190
276929910188,7cyclictest0-21swapper/220:29:282
285032630,5sleep12768599cyclictest19:10:181
27692996157,3cyclictest0-21swapper/220:58:192
323092480,6sleep230-21rcuc/221:45:132
21672470,2sleep10-21swapper/100:15:261
27679993812,13cyclictest11053-21munin-run23:25:010
2769999321,11cyclictest808-21snmpd19:20:143
2769999311,13cyclictest0-21swapper/321:05:023
2769999310,13cyclictest0-21swapper/322:09:333
2769999301,11cyclictest0-21swapper/320:10:333
2769999300,23cyclictest0-21swapper/323:58:013
2769999300,13cyclictest0-21swapper/300:07:133
2769999300,13cyclictest0-21swapper/300:07:133
2769999300,11cyclictest0-21swapper/321:12:223
2768599301,3cyclictest21078-21gltestperf20:05:171
2767999309,17cyclictest3213-21munin-run20:40:010
2769999291,13cyclictest0-21swapper/322:33:213
2769999291,12cyclictest0-21swapper/321:20:103
2769999291,11cyclictest0-21swapper/323:52:103
2769999290,22cyclictest0-21swapper/321:59:173
2768599291,10cyclictest0-21swapper/123:04:531
27679992917,7cyclictest8816-40dnf19:35:290
2769999281,9cyclictest0-21swapper/300:04:133
2769999281,17cyclictest0-21swapper/322:22:013
2769999281,13cyclictest0-21swapper/321:38:133
2769999281,12cyclictest0-21swapper/322:48:383
2769999281,10cyclictest0-21swapper/322:13:053
2769999281,10cyclictest0-21swapper/300:33:173
2769999281,10cyclictest0-21swapper/300:10:373
2769999280,21cyclictest0-21swapper/322:55:343
2769999280,21cyclictest0-21swapper/322:55:343
2769999280,17cyclictest0-21swapper/321:32:213
2769999280,13cyclictest4420-21grep20:40:183
2769999280,13cyclictest0-21swapper/322:39:013
2769999280,11cyclictest0-21swapper/321:50:573
2767999284,11cyclictest17038-21idleruntime-cro22:25:010
2767999281,14cyclictest985-21munin-run20:35:010
2767999281,14cyclictest985-21munin-run20:35:010
2767999281,13cyclictest19225-21munin-run22:30:010
2769999271,16cyclictest0-21swapper/319:43:053
2769999271,13cyclictest0-21swapper/322:44:093
2769999271,12cyclictest0-21swapper/321:48:493
2769999271,12cyclictest0-21swapper/320:24:533
2769999271,10cyclictest0-21swapper/322:02:573
2769999271,10cyclictest0-21swapper/321:42:493
2769999271,10cyclictest0-21swapper/320:09:213
2769999270,16cyclictest0-21swapper/323:23:543
2769999270,11cyclictest0-21swapper/300:29:053
2769999270,10cyclictest0-21swapper/320:35:583
2767999273,16cyclictest1856-21/usr/sbin/munin21:50:230
2767999272,12cyclictest12001-21munin-run21:00:010
147962271,22sleep115015-21/usr/sbin/munin21:05:161
2769999267,3cyclictest0-21swapper/323:45:583
2769999261,9cyclictest0-21swapper/323:43:023
2769999261,10cyclictest0-21swapper/320:47:583
2769999260,18cyclictest0-21swapper/319:53:453
2769999260,12cyclictest0-21swapper/323:26:543
2769999260,11cyclictest0-21swapper/319:46:293
27692992619,4cyclictest5028-21diskstats23:10:162
2768599262,12cyclictest8816-40dnf19:35:301
2768599261,7cyclictest0-21swapper/122:23:141
2767999265,13cyclictest3790-21hddtemp_smartct00:20:170
2767999263,13cyclictest32637-21/usr/sbin/munin23:00:140
42952250,8sleep230-21rcuc/221:55:122
2769999256,3cyclictest0-21swapper/321:08:083
2769999251,9cyclictest0-21swapper/322:18:213
2769999251,11cyclictest0-21swapper/323:01:063
2769999251,10cyclictest0-21swapper/321:16:183
2769999251,10cyclictest0-21swapper/320:27:503
2769999250,9cyclictest0-21swapper/323:15:343
2769999250,9cyclictest0-21swapper/319:14:093
2769999250,11cyclictest0-21swapper/319:18:343
2769999250,10cyclictest0-21swapper/323:39:343
2769999250,10cyclictest0-21swapper/323:07:243
2769999250,10cyclictest0-21swapper/320:50:423
2768599251,9cyclictest0-21swapper/123:35:491
27685992512,11cyclictest5484-21kworker/u8:019:20:131
2768599251,18cyclictest0-21swapper/121:42:571
2768599251,17cyclictest0-21swapper/121:45:241
2768599251,13cyclictest0-21swapper/122:33:451
2767999255,8cyclictest26353-21grep20:15:270
2767999253,11cyclictest0-21swapper/022:00:250
84612243,3sleep18462-21/usr/sbin/munin22:05:121
2769999249,3cyclictest0-21swapper/319:36:013
2769999248,2cyclictest0-21swapper/320:18:573
2769999245,3cyclictest0-21swapper/300:15:533
2769999241,9cyclictest0-21swapper/319:57:333
2769999241,9cyclictest0-21swapper/300:35:533
2769999241,10cyclictest0-21swapper/320:31:463
2769999241,10cyclictest0-21swapper/320:31:463
27699992410,3cyclictest0-21swapper/320:00:053
2769999240,11cyclictest0-21swapper/321:25:343
2769299241,10cyclictest0-21swapper/200:25:012
2768599249,13cyclictest18849-21kworker/u8:022:44:321
27685992418,3cyclictest17441-21irqstats21:10:181
27685992413,8cyclictest19679-21grep21:15:191
2767999245,11cyclictest23593-21sed23:50:270
2767999244,10cyclictest0-21swapper/023:45:190
2767999242,19cyclictest19692-21unixbench_multi20:00:230
2767999242,12cyclictest16400-21diskmemload21:30:290
2767999241,8cyclictest27248-21gltestperf00:00:170
2767999241,12cyclictest4467-21idleruntime-cro23:10:010
2767999241,10cyclictest29466-21gltestperf00:05:170
2767999241,10cyclictest29466-21gltestperf00:05:170
2767999241,10cyclictest27948-21expr21:35:150
7232230,4sleep011-21rcuc/019:20:150
2769999236,3cyclictest0-21swapper/323:31:003
2769999231,8cyclictest0-21swapper/320:55:463
2769999231,12cyclictest0-21swapper/319:32:493
2769999230,11cyclictest0-21swapper/319:25:053
2769999230,11cyclictest0-21swapper/300:22:173
2769999230,10cyclictest0-21swapper/323:13:423
2768599231,6cyclictest0-21swapper/123:57:261
2768599231,19cyclictest2498-21smart_sda20:35:241
27685992310,11cyclictest23743-21kworker/u8:119:29:581
2767999232,9cyclictest10062-21seq22:05:260
2767999232,10cyclictest15801-21cpuspeed_turbos23:35:130
2767999231,6cyclictest0-21swapper/021:25:170
27679992316,4cyclictest21885-21expr22:35:140
27679992316,4cyclictest11538-40dnf00:38:230
2767999231,11cyclictest1627-21munin-run21:50:020
2767999230,13cyclictest10659-21/usr/sbin/munin22:10:230
2767999230,12cyclictest0-21swapper/022:45:130
2769999220,10cyclictest0-21swapper/322:28:333
2769299228,4cyclictest17827-21seq19:58:202
2769299228,4cyclictest0-21swapper/221:42:432
2769299221,13cyclictest0-21swapper/220:44:592
2768599228,12cyclictest18849-21kworker/u8:022:35:251
2768599222,10cyclictest26763-21sh20:20:001
2768599221,6cyclictest0-21swapper/123:46:491
2768599221,3cyclictest11415-21cat22:10:181
2768599221,18cyclictest6445-21irqstats20:45:191
2768599221,18cyclictest22997-21munin-run21:25:001
2768599221,15cyclictest0-21swapper/123:22:481
2767999228,9cyclictest0-21swapper/022:55:180
2767999228,9cyclictest0-21swapper/022:55:180
2767999223,11cyclictest0-21swapper/021:40:120
2767999222,12cyclictest6881-21/usr/sbin/munin23:15:180
2767999221,5cyclictest0-21swapper/022:50:170
27679992213,6cyclictest15798-21cat19:55:020
86412211,2sleep20-21swapper/223:17:322
2769299211,5cyclictest0-21swapper/222:06:332
2768599219,3cyclictest31444-21cpu20:30:121
2768599219,3cyclictest31444-21cpu20:30:121
2768599219,10cyclictest5484-21kworker/u8:019:19:571
27685992117,3cyclictest0-21swapper/100:07:111
27685992117,3cyclictest0-21swapper/100:07:111
2768599211,5cyclictest0-21swapper/123:09:541
2768599211,4cyclictest23559-21df_inode21:25:111
27685992114,4cyclictest30574-21grep20:25:261
2768599211,18cyclictest0-21swapper/121:35:591
2768599211,18cyclictest0-21swapper/100:10:281
2768599211,14cyclictest0-21swapper/123:15:291
2768599211,14cyclictest0-21swapper/100:25:081
2768599211,11cyclictest8290-21cat22:05:021
2768599211,11cyclictest3446-21users21:50:271
2768599211,11cyclictest29719-21grep22:50:311
2768599211,11cyclictest17596-21diskstats22:25:131
2768599211,10cyclictest28650-21munin-run00:05:011
2768599210,16cyclictest0-21swapper/100:36:481
2767999218,2cyclictest12569-21sh21:00:150
2767999213,8cyclictest0-21swapper/019:30:150
2767999213,7cyclictest0-21swapper/000:15:050
2767999212,7cyclictest7923-21tr20:50:120
2767999211,7cyclictest0-21swapper/021:24:180
2767999211,10cyclictest13257-21perf23:30:020
2767999210,10cyclictest15449-21awk23:35:020
191772211,2sleep219362-21grep21:15:162
2769299204,9cyclictest0-21swapper/221:36:092
2769299203,9cyclictest0-21swapper/221:32:142
2769299202,5cyclictest0-21swapper/222:10:252
2769299201,7cyclictest0-21swapper/200:00:052
2769299201,5cyclictest0-21swapper/200:12:072
2769299201,16cyclictest30862-21cut22:55:192
2769299201,16cyclictest30862-21cut22:55:192
2768599207,6cyclictest0-21swapper/123:43:371
2768599203,8cyclictest7519-21/usr/sbin/munin00:30:321
2768599202,4cyclictest0-21swapper/120:20:281
27685992013,2cyclictest0-21swapper/122:47:281
2768599201,18cyclictest0-21swapper/121:33:431
2768599201,17cyclictest0-21swapper/120:10:461
2768599201,17cyclictest0-21swapper/119:53:421
2768599201,17cyclictest0-21swapper/119:40:261
2768599201,14cyclictest0-21swapper/123:13:591
2768599201,14cyclictest0-21swapper/122:56:141
2768599201,14cyclictest0-21swapper/122:56:141
2768599201,12cyclictest0-21swapper/100:20:251
2768599201,11cyclictest17221-21ps19:55:261
2768599201,11cyclictest14804-21chrt22:19:561
2768599201,10cyclictest23589-21cat23:50:261
2768599201,10cyclictest11670-21grep19:45:141
2767999208,4cyclictest19738-21NetworkManager23:43:560
2767999205,3cyclictest3430-21/usr/sbin/munin20:40:110
2767999201,4cyclictest87350irq/126-enp1s0-20:05:160
27679992013,4cyclictest28702-21/usr/sbin/munin19:10:190
27679992012,5cyclictest14113-21smart_sda22:15:240
27679992011,6cyclictest827-21runrttasks19:25:260
27679992011,6cyclictest11610-21/usr/sbin/munin19:45:170
2767999201,11cyclictest87550irq/128-enp1s0-00:10:230
2767999201,11cyclictest24457-21/usr/sbin/munin23:55:160
2767999201,11cyclictest0-21swapper/000:30:110
2769299198,3cyclictest996-21awk20:35:022
2769299198,3cyclictest996-21awk20:35:022
2769299195,4cyclictest9180-21sh19:40:012
2769299194,8cyclictest12050irq/123-ahci[0000:00:012
2769299194,3cyclictest87350irq/126-enp1s0-22:30:142
2769299193,8cyclictest0-21swapper/200:28:072
2769299192,6cyclictest14212-21munin-run21:05:022
2769299191,7cyclictest0-21swapper/221:25:112
27692991912,4cyclictest20517-21cpuspeed_turbos20:05:132
2769299191,11cyclictest0-21swapper/223:29:322
2769299191,11cyclictest0-21swapper/222:21:292
2768599193,10cyclictest8833-21kworker/u8:023:33:201
2768599191,9cyclictest7846-21cpu20:50:121
27685991913,3cyclictest19241-21ntpq20:00:191
2768599191,16cyclictest0-21swapper/123:25:021
2768599191,16cyclictest0-21swapper/120:56:161
2768599190,5cyclictest0-21swapper/120:44:581
2767999199,7cyclictest5986-21hddtemp_smartct00:25:170
27679991913,4cyclictest17250-21mailstats19:55:260
27679991911,5cyclictest30480-21smart_sda19:15:030
27679991911,5cyclictest20705-21smart_sda22:30:230
2767999191,13cyclictest0-21swapper/022:43:220
2767999191,11cyclictest16188-21sshd21:07:270
27679991910,6cyclictest909-21sssd_nss21:55:240
27679991910,6cyclictest5731-21/usr/sbin/munin23:10:220
171682190,3sleep00-21swapper/021:10:150
35672181,2sleep20-21swapper/221:51:482
2769299185,4cyclictest32-21ksoftirqd/221:05:192
2769299184,10cyclictest87550irq/128-enp1s0-00:18:052
2769299183,9cyclictest0-21swapper/223:07:562
2769299181,9cyclictest0-21swapper/222:45:302
2769299181,8cyclictest28875-21/usr/sbin/munin00:05:232
2769299181,8cyclictest28875-21/usr/sbin/munin00:05:232
2769299181,7cyclictest17701-21ntp_states21:10:222
2769299181,6cyclictest23103-21unixbench-2d22:35:272
2769299181,6cyclictest0-21swapper/223:35:132
2769299181,4cyclictest30005-21munin-node19:15:042
2769299181,15cyclictest0-21swapper/220:15:092
27692991811,4cyclictest14612-21awk23:30:212
2768599181,16cyclictest0-21swapper/119:33:451
2768599180,17cyclictest0-21swapper/121:04:551
2767999189,6cyclictest6403-21/usr/sbin/munin20:45:180
2767999189,6cyclictest26996-21/usr/sbin/munin20:20:200
2767999185,10cyclictest30940-21runrttasks20:27:140
2767999181,6cyclictest0-21swapper/021:16:130
2767999181,15cyclictest0-21swapper/020:13:420
240172181,2sleep20-21swapper/223:52:282
197592181,2sleep20-21swapper/223:44:132
2769299173,3cyclictest7612-21awk20:50:022
2769299172,9cyclictest0-21swapper/222:27:142
2769299171,9cyclictest0-21swapper/222:15:222
2769299171,9cyclictest0-21swapper/219:27:082
2769299171,7cyclictest0-21swapper/222:00:222
2769299171,6cyclictest14616-21irqstats19:50:182
2769299171,6cyclictest0-21swapper/221:23:592
2769299171,4cyclictest0-21swapper/200:30:182
27692991713,3cyclictest0-21swapper/223:46:462
27692991712,3cyclictest10475-21if_enp2s000:35:172
27692991712,2cyclictest23839-21/usr/sbin/munin22:40:212
2769299171,15cyclictest0-21swapper/219:47:112
27692991711,4cyclictest9739-21cut23:20:142
27692991711,4cyclictest2803-21grep20:35:272
2769299171,10cyclictest0-21swapper/219:40:102
27692991710,4cyclictest27139-21cpuspeed_turbos20:20:142
2769299170,6cyclictest16400-21diskmemload23:00:152
27685991714,2cyclictest0-21swapper/121:56:341
2769299166,2cyclictest30-21rcuc/220:50:202
2769299161,6cyclictest790-21in:imjournal19:24:272
2769299161,6cyclictest23592-21grep20:10:202
27692991612,3cyclictest0-21swapper/219:30:462
27692991611,3cyclictest28988-21if_enp1s022:50:232
27692991611,3cyclictest28958-21ntp_states19:10:242
27692991611,3cyclictest19891-21sshd20:01:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional