You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 08:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Aug 15, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226652240164,24sleep00-21swapper/019:07:060
227972237162,24sleep20-21swapper/219:08:492
226042231162,56sleep10-21swapper/119:06:181
228092206169,25sleep30-21swapper/319:08:583
2820121680,4sleep02296999cyclictest21:45:180
685021480,3sleep02296999cyclictest00:38:470
2298199103100,2cyclictest0-21swapper/200:38:152
30502620,1sleep20-21swapper/220:50:022
184362620,2sleep081ktimersoftd/022:35:260
256442560,2sleep22298199cyclictest00:05:262
143792550,1sleep10-21swapper/121:15:131
22981994845,2cyclictest0-21swapper/223:33:002
293222390,2sleep230-21rcuc/219:20:202
22988993727,3cyclictest0-21swapper/319:37:353
2298899361,28cyclictest0-21swapper/320:48:223
22981993532,2cyclictest0-21swapper/223:26:482
22969993510,11cyclictest28667-21awk20:35:010
2298899330,15cyclictest0-21swapper/323:48:273
2297699332,25cyclictest27918-21kworker/u8:023:19:141
2296999339,11cyclictest582-21idleruntime-cro20:45:010
2296999339,11cyclictest582-21idleruntime-cro20:45:010
2296999333,14cyclictest5087-21munin-run20:55:010
2298899321,29cyclictest0-21swapper/300:20:513
2298899321,28cyclictest0-21swapper/323:41:083
2298899321,21cyclictest0-21swapper/319:15:393
2298899321,15cyclictest0-21swapper/320:36:433
2298899321,14cyclictest0-21swapper/321:13:113
2297699320,25cyclictest3944-21kworker/u8:200:30:071
2296999324,13cyclictest18295-21awk21:25:010
2298899311,20cyclictest0-21swapper/319:33:353
2298899311,20cyclictest0-21swapper/319:33:353
22988993017,7cyclictest0-21swapper/320:13:153
22988993016,7cyclictest0-21swapper/320:32:073
2298899301,20cyclictest0-21swapper/320:56:273
2298899301,20cyclictest0-21swapper/319:13:313
2298899301,19cyclictest0-21swapper/320:43:513
2298899301,19cyclictest0-21swapper/320:43:503
2298899301,13cyclictest0-21swapper/323:23:233
2298899301,13cyclictest0-21swapper/319:26:353
2298899301,12cyclictest0-21swapper/321:28:353
2298899301,11cyclictest0-21swapper/323:02:193
2298899300,12cyclictest0-21swapper/321:56:033
2297699302,24cyclictest2568-21munin-run00:30:011
2297699301,22cyclictest0-21swapper/122:53:391
22988992915,3cyclictest0-21swapper/319:40:393
2298899291,27cyclictest0-21swapper/321:40:233
2298899291,19cyclictest0-21swapper/321:02:193
2298899291,13cyclictest0-21swapper/322:42:233
2298899291,13cyclictest0-21swapper/321:48:393
2298899291,12cyclictest0-21swapper/320:15:033
2298899291,10cyclictest0-21swapper/321:32:353
2298899290,15cyclictest0-21swapper/322:39:073
2298899290,13cyclictest0-21swapper/321:20:253
2298899290,12cyclictest0-21swapper/322:19:393
2297699291,22cyclictest0-21swapper/123:24:191
2298899281,18cyclictest0-21swapper/322:45:233
2298899281,14cyclictest0-21swapper/323:05:473
2298899281,13cyclictest0-21swapper/320:05:273
2298899281,12cyclictest0-21swapper/322:31:393
2298899281,12cyclictest0-21swapper/300:17:153
2298899281,11cyclictest0-21swapper/323:38:383
2298899280,13cyclictest0-21swapper/323:13:113
2298899280,11cyclictest0-21swapper/322:27:073
2298899280,11cyclictest0-21swapper/322:22:393
2298199285,3cyclictest0-21swapper/220:17:442
2297699282,4cyclictest26143-21munin-run00:10:011
2297699281,22cyclictest0-21swapper/122:44:461
2298899271,9cyclictest0-21swapper/323:28:593
2298899271,7cyclictest0-21swapper/300:26:093
2298899271,13cyclictest0-21swapper/321:16:473
2298899271,12cyclictest0-21swapper/322:59:473
2298899271,12cyclictest0-21swapper/300:35:153
2298899271,11cyclictest11682-21diskmemload23:52:153
2298899271,11cyclictest0-21swapper/300:11:513
22976992724,2cyclictest0-21swapper/120:45:061
2297699271,6cyclictest0-21swapper/122:23:551
2297699271,19cyclictest0-21swapper/122:05:551
2296999275,11cyclictest0-21swapper/020:10:020
2296999274,11cyclictest15480-21awk20:05:010
2298899261,8cyclictest0-21swapper/319:56:313
2298899261,14cyclictest0-21swapper/300:30:153
2298899261,12cyclictest0-21swapper/320:51:113
2298899261,11cyclictest0-21swapper/323:58:233
2298899260,10cyclictest0-21swapper/323:16:413
2297699262,3cyclictest3517-21munin-run22:05:011
2297699262,21cyclictest7226-21hddtemp_smartct23:25:151
2297699262,15cyclictest0-21swapper/121:28:141
2297699262,15cyclictest0-21swapper/121:14:111
2296999260,17cyclictest4504-21kworker/0:100:32:450
2296999260,10cyclictest8494-21kworker/0:219:47:370
25492250,2sleep00-21swapper/019:35:110
22988992511,3cyclictest23989-21sshd20:21:393
2298899251,13cyclictest0-21swapper/319:20:513
2298899251,12cyclictest0-21swapper/321:35:033
2298899251,12cyclictest0-21swapper/321:09:473
2298899251,11cyclictest0-21swapper/323:32:433
2298899251,11cyclictest0-21swapper/322:54:153
2298899251,11cyclictest0-21swapper/322:13:353
22988992510,9cyclictest0-21swapper/320:27:393
2298899250,13cyclictest0-21swapper/321:52:073
2298899250,13cyclictest0-21swapper/321:52:073
2297699252,21cyclictest0-21swapper/119:38:271
2297699252,15cyclictest0-21swapper/122:45:421
22976992513,7cyclictest0-21swapper/122:19:531
2297699251,21cyclictest1816-21sshd23:13:411
2297699251,21cyclictest17657-21latency_hist20:10:021
2297699251,16cyclictest0-21swapper/121:21:161
2297699251,12cyclictest32736-21awk00:25:011
2297699251,11cyclictest28351-21cpuspeed_turbos19:20:121
22976992510,3cyclictest25854-21munin-node19:15:031
2297699250,16cyclictest0-21swapper/123:40:441
2296999252,10cyclictest30590-21kworker/0:220:56:050
2296999251,7cyclictest9785-21kworker/0:122:30:130
22988992416,3cyclictest0-21swapper/320:00:553
2298899241,12cyclictest0-21swapper/322:08:553
2298899241,12cyclictest0-21swapper/319:52:193
2298899241,12cyclictest0-21swapper/300:08:153
2298899241,11cyclictest0-21swapper/322:02:033
2298899241,11cyclictest0-21swapper/300:02:273
2298899241,10cyclictest0-21swapper/319:47:273
2298199248,2cyclictest0-21swapper/220:27:532
2297699249,4cyclictest0-21swapper/100:10:221
2297699249,3cyclictest20763-21/usr/sbin/munin21:30:101
2297699241,9cyclictest0-21swapper/122:32:001
2297699241,6cyclictest0-21swapper/123:00:021
22976992413,3cyclictest0-21swapper/123:35:521
22976992413,3cyclictest0-21swapper/121:36:491
2297699241,20cyclictest30185-21egrep23:05:121
2297699241,20cyclictest27880-21cut23:00:011
2297699241,20cyclictest19773-21cpu23:55:101
2297699241,20cyclictest16602-21smart_sda23:45:241
2297699241,20cyclictest129472sleep119:56:101
2297699241,14cyclictest0-21swapper/123:33:141
2297699241,14cyclictest0-21swapper/122:39:431
2297699241,12cyclictest0-21swapper/123:50:141
2297699241,11cyclictest7601-21tr21:00:131
2297699241,11cyclictest29229-21diskstats20:35:161
2297699241,10cyclictest27529-21cat21:45:121
22976992410,5cyclictest0-21swapper/121:53:181
22976992410,5cyclictest0-21swapper/121:53:181
2297699240,21cyclictest32275-21processes20:40:231
2297699240,21cyclictest32275-21processes20:40:231
2296999245,16cyclictest21093-21munin-run22:45:010
2296999242,9cyclictest10715-21kworker/0:019:51:410
2296999242,7cyclictest3761-21kworker/0:123:40:130
2296999241,11cyclictest11700-21perf21:10:010
233852230,3sleep230-21rcuc/222:50:002
2298199231,8cyclictest27695-21munin-run23:00:012
2297699239,3cyclictest29371-21irqstats00:15:181
2297699239,3cyclictest22869-21cat00:00:201
2297699239,3cyclictest14647-21latency_hist22:30:021
2297699237,3cyclictest24624-21kworker/u8:021:58:081
2297699231,20cyclictest0-21swapper/119:30:201
2297699231,20cyclictest0-21swapper/119:30:201
2297699231,14cyclictest0-21swapper/121:40:401
2297699231,11cyclictest19056-21proc_pri20:10:231
2297699231,10cyclictest28656-21pluginstate20:35:011
2296999231,21cyclictest2252-21kworker/0:019:43:530
2296999230,12cyclictest27415-21kworker/0:223:03:010
82042220,2sleep20-21swapper/221:00:172
2298199224,9cyclictest12050irq/123-ahci[0020:10:152
2298199221,8cyclictest5940-21gltestperf20:55:162
2297699229,2cyclictest25747-21smart_sda20:25:231
2297699226,4cyclictest23-21ksoftirqd/122:10:231
2297699221,19cyclictest0-21swapper/119:26:131
2297699221,19cyclictest0-21swapper/119:14:561
2297699221,18cyclictest6373-21ntpq20:55:211
2297699221,10cyclictest22143-21cat20:20:011
2296999222,8cyclictest32060-21kworker/0:023:10:510
2296999222,8cyclictest23803-21kworker/0:100:13:530
2296999222,7cyclictest19836-21kworker/0:220:16:530
2296999221,9cyclictest9785-21kworker/0:122:29:210
2296999221,5cyclictest0-21swapper/021:33:560
40132211,3sleep00-21swapper/023:18:410
27492210,3sleep2311ktimersoftd/223:15:132
2298199217,4cyclictest22961-21cyclictest22:31:342
2298199214,9cyclictest23238-21cpuspeed_turbos19:10:102
2298199211,7cyclictest5607-21memory19:40:192
2298199211,7cyclictest11093-21if_enp2s022:20:162
2298199211,5cyclictest24624-21kworker/u8:021:40:242
2298199211,12cyclictest0-21swapper/223:10:132
2297699217,13cyclictest2234-21kworker/u8:021:07:471
22976992117,3cyclictest0-21swapper/119:48:101
2297699211,18cyclictest0-21swapper/119:52:241
2297699211,11cyclictest0-21swapper/120:20:181
2297699210,2cyclictest87550irq/128-enp1s0-00:35:591
2297699210,20cyclictest0-21swapper/120:54:051
2296999212,9cyclictest9785-21kworker/0:122:22:530
2296999211,7cyclictest14770-21kworker/0:223:45:170
22969992111,3cyclictest87550irq/128-enp1s0-22:13:010
2296999211,12cyclictest14108-21/usr/sbin/munin21:15:220
2296999210,10cyclictest19875-21awk20:15:020
9352200,2sleep00-21swapper/021:55:420
2298199204,8cyclictest12050irq/123-ahci[0019:55:182
2298199202,6cyclictest30545-21ntp_kernel_pll_21:50:202
2298199202,6cyclictest30545-21ntp_kernel_pll_21:50:202
2298199201,7cyclictest6389-21cut22:10:152
2298199201,6cyclictest8988-21cat19:50:022
2298199201,6cyclictest22886-21date21:35:012
2298199201,6cyclictest18072-21taskset21:22:322
2298199201,6cyclictest12258-21diskstats21:10:142
2298199201,4cyclictest0-21swapper/222:50:152
22981992014,3cyclictest3166-21cut19:35:152
2298199201,3cyclictest0-21swapper/200:21:002
2298199201,11cyclictest28283-21forks23:00:152
2298199201,11cyclictest2792-21/usr/sbin/munin00:30:192
2298199201,10cyclictest32049-21users19:25:272
2297699208,2cyclictest0-21swapper/120:00:131
2297699201,18cyclictest0-21swapper/119:41:211
62942191,2sleep00-21swapper/023:24:430
277012191,2sleep20-21swapper/219:15:522
2298199191,9cyclictest9088-21/usr/sbin/munin19:50:142
2298199191,9cyclictest18145-21expr23:50:152
2298199191,8cyclictest0-21swapper/220:30:212
2298199191,7cyclictest27303-21/usr/sbin/munin21:45:172
2298199191,7cyclictest19991-21expr23:55:102
2298199191,7cyclictest0-21swapper/222:04:272
2298199191,6cyclictest493-21/usr/sbin/munin00:25:172
2298199191,6cyclictest19569-21ntp_states21:25:192
2298199191,6cyclictest15733-21tune2fs23:45:152
2298199191,6cyclictest0-21swapper/223:40:262
2298199191,5cyclictest0-21swapper/221:38:472
2298199191,4cyclictest27502-21pmu-power00:10:212
2298199191,12cyclictest0-21swapper/223:06:562
2298199191,10cyclictest29318-21if_enp2s000:15:172
2298199191,10cyclictest15691-21/usr/sbin/munin20:05:202
2298199190,9cyclictest0-21swapper/200:00:242
2298199190,11cyclictest0-21swapper/222:08:092
2298199190,10cyclictest20295-21ps22:40:212
2296999198,2cyclictest21310-21kworker/0:022:45:170
2296999192,8cyclictest3761-21kworker/0:123:34:090
2296999192,8cyclictest24459-21kworker/0:019:20:410
2296999191,7cyclictest2206-21kworker/0:000:29:410
2296999191,4cyclictest0-21swapper/022:08:190
2296999191,3cyclictest0-21swapper/023:28:150
2296999191,3cyclictest0-21swapper/021:14:070
22969991913,3cyclictest24121-21grep22:50:150
2296999191,12cyclictest0-21swapper/023:55:110
2296999191,10cyclictest808-21snmpd19:59:340
2296999190,8cyclictest28260-21kworker/0:000:17:130
2298199185,3cyclictest87350irq/126-enp1s0-20:40:162
2298199185,3cyclictest87350irq/126-enp1s0-20:40:162
2298199184,7cyclictest87650irq/129-enp1s0-20:00:172
2298199182,8cyclictest87650irq/129-enp1s0-20:54:412
2298199181,5cyclictest827-21runrttasks22:15:522
2298199181,5cyclictest22283-21/usr/sbin/munin20:20:222
2298199181,5cyclictest14964-21latency21:15:182
2298199181,5cyclictest13597-21ntp_states22:25:222
2298199181,5cyclictest0-21swapper/223:39:582
22981991813,3cyclictest5403-21ntp_kernel_pll_23:20:202
2296999188,3cyclictest26025-21kworker/0:200:09:490
2296999185,5cyclictest755-21rngd21:25:160
2296999181,9cyclictest0-21swapper/021:35:320
2296999181,5cyclictest0-21swapper/022:03:440
22969991812,5cyclictest0-21swapper/020:25:240
2296999180,5cyclictest3944-21kworker/u8:200:02:070
10572181,2sleep20-21swapper/221:57:172
278482170,2sleep00-21swapper/019:17:420
2298199172,4cyclictest17014-21cpuspeed_turbos22:35:122
2298199171,6cyclictest0-21swapper/221:05:232
2298199171,4cyclictest29795-21memory20:35:212
22969991712,3cyclictest7837-21df21:00:140
22969991712,3cyclictest25721-21/usr/sbin/munin22:55:180
2296999171,13cyclictest30125-21/usr/sbin/munin23:05:220
22969991711,3cyclictest29645-21grep21:50:120
22969991711,3cyclictest29645-21grep21:50:120
22969991710,4cyclictest25753-21tail21:40:150
2298199163,3cyclictest87350irq/126-enp1s0-19:30:102
2298199163,3cyclictest87350irq/126-enp1s0-19:30:102
2296999166,3cyclictest87350irq/126-enp1s0-19:10:220
2296999161,5cyclictest9200-21ntp_states22:15:200
2296999161,4cyclictest28881-21/usr/sbin/munin20:35:170
22969991612,3cyclictest0-21swapper/019:30:050
22969991612,3cyclictest0-21swapper/019:30:040
22969991612,3cyclictest0-21swapper/019:28:530
2296999161,13cyclictest0-21swapper/020:45:040
2296999161,11cyclictest3944-21kworker/u8:223:50:230
22969991610,3cyclictest11394-21fschecks_count23:35:160
2296999151,6cyclictest808-21snmpd20:24:390
22969991510,3cyclictest827-21runrttasks00:20:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional