You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 00:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Dec 23, 2025 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204012238162,25sleep00-21swapper/007:09:590
204012238162,25sleep00-21swapper/007:09:590
201702237164,26sleep30-21swapper/307:07:003
201702237164,26sleep30-21swapper/307:07:003
203712236162,24sleep20-21swapper/207:09:372
203712236162,24sleep20-21swapper/207:09:372
200602226153,24sleep10-21swapper/107:05:341
200602226153,24sleep10-21swapper/107:05:341
1628321380,6sleep02047599cyclictest09:25:150
1628321380,6sleep02047599cyclictest09:25:150
16162950,5sleep12047999cyclictest11:20:011
119662510,1sleep30-21swapper/308:00:183
53512460,1sleep20-21swapper/211:25:242
20475993411,11cyclictest9823-21awk10:25:010
204212330,5sleep22048799cyclictest12:00:202
2047999310,28cyclictest0-21swapper/109:50:251
2047599302,13cyclictest14205-21munin-run10:35:010
20487992926,2cyclictest0-21swapper/209:35:572
2047999291,26cyclictest0-21swapper/112:36:581
2047999291,26cyclictest0-21swapper/110:20:221
2048799281,7cyclictest0-21swapper/210:05:202
2047999281,25cyclictest0-21swapper/111:06:061
2047999281,25cyclictest0-21swapper/109:05:101
2047999281,25cyclictest0-21swapper/108:40:141
2047999281,25cyclictest0-21swapper/108:16:461
2047999281,25cyclictest0-21swapper/107:12:371
2047999280,26cyclictest0-21swapper/108:50:151
2049399271,9cyclictest0-21swapper/312:00:183
2049399271,9cyclictest0-21swapper/309:38:523
2047999271,4cyclictest1370-21cpuspeed_turbos10:05:121
20479992714,10cyclictest31337-21latency_hist10:00:021
2047999270,25cyclictest0-21swapper/110:19:381
2047999270,25cyclictest0-21swapper/109:35:171
2047999270,25cyclictest0-21swapper/108:45:221
2047599271,6cyclictest0-21swapper/011:40:180
2049399263,16cyclictest0-21swapper/310:52:323
2049399261,6cyclictest0-21swapper/310:00:543
2047999264,20cyclictest8510-21cpu11:35:161
2047999261,23cyclictest0-21swapper/112:23:341
2047999260,24cyclictest0-21swapper/110:38:591
2049399251,8cyclictest0-21swapper/309:26:483
2049399251,8cyclictest0-21swapper/309:26:483
2049399251,5cyclictest24841-21kworker/u8:011:46:403
2049399251,12cyclictest0-21swapper/310:05:033
2047999254,18cyclictest27513-21unixbench-2d12:15:261
2047999251,23cyclictest0-21swapper/111:54:531
2047999251,23cyclictest0-21swapper/110:46:241
2047999251,22cyclictest0-21swapper/111:45:221
2047999251,22cyclictest0-21swapper/110:28:431
2047999251,15cyclictest0-21swapper/108:10:171
2047599251,8cyclictest0-21swapper/012:25:480
2047599250,23cyclictest0-21swapper/008:15:210
2049399246,10cyclictest9193-21diskmemload11:24:103
2049399245,8cyclictest87250irq/129-enp1s0-11:10:123
2049399241,6cyclictest0-21swapper/310:19:023
2049399241,5cyclictest7974-21kworker/u8:211:32:563
2048799241,7cyclictest0-21swapper/210:01:162
2047999247,14cyclictest9496-21idleruntime07:55:171
20479992418,3cyclictest22550-21perl08:25:161
2047999241,21cyclictest0-21swapper/112:26:071
2047999240,4cyclictest0-21swapper/110:44:331
2047999240,4cyclictest0-21swapper/110:44:331
2047599241,6cyclictest0-21swapper/011:34:410
2047599241,10cyclictest27386-21sshd07:20:320
104572240,3sleep210458-21awk09:10:202
2049399231,4cyclictest14487-21kworker/u8:112:15:263
2049399231,4cyclictest0-21swapper/310:43:493
2049399231,4cyclictest0-21swapper/310:43:493
2049399230,12cyclictest24398-21kworker/u8:109:50:183
2048799232,5cyclictest986-1kworker/u9:112:16:442
2048799231,5cyclictest0-21swapper/209:30:032
2047999239,3cyclictest53532switchtime11:25:241
2047999231,3cyclictest0-21swapper/108:00:201
2047999231,2cyclictest7851-21smart_sda07:50:221
2047999231,21cyclictest0-21swapper/111:14:591
20479992311,3cyclictest0-21swapper/112:30:241
2047999231,11cyclictest2448-21cat07:40:121
2047999230,4cyclictest0-21swapper/112:03:581
2047599235,5cyclictest552-21sshd10:00:320
2047599232,10cyclictest18822-21/usr/sbin/munin10:45:140
20475992314,6cyclictest14122-21cat11:45:250
2047599231,10cyclictest1864-21users12:30:270
2047599230,10cyclictest0-21swapper/010:15:260
116312230,2sleep30-21swapper/309:15:103
2049399225,8cyclictest12050irq/123-ahci[0007:55:003
2049399224,10cyclictest9193-21diskmemload12:37:363
2049399223,10cyclictest8903-21ntp_states10:20:223
2049399223,10cyclictest0-21swapper/311:25:213
2049399221,6cyclictest0-21swapper/311:16:393
2049399221,6cyclictest0-21swapper/310:56:003
2049399221,6cyclictest0-21swapper/310:37:213
2049399221,3cyclictest14487-21kworker/u8:112:13:383
2049399221,12cyclictest0-21swapper/311:35:243
2048799228,2cyclictest3048-21kworker/2:011:55:162
2048799228,2cyclictest3048-21kworker/2:011:55:162
2048799221,6cyclictest0-21swapper/211:36:572
2048799221,5cyclictest0-21swapper/211:03:382
20487992211,3cyclictest30-21rcuc/209:40:232
20479992217,3cyclictest4968-21wc09:00:021
20479992217,3cyclictest28633-21gltestperf07:25:141
2047999221,3cyclictest24528-21df08:30:131
2047999221,20cyclictest0-21swapper/110:53:471
2047999221,18cyclictest2196-21df11:20:141
2047599222,8cyclictest27626-21/usr/sbin/munin11:05:250
2047599221,6cyclictest24190-21/usr/sbin/munin08:30:110
20475992216,3cyclictest3373-21grep11:20:260
2047599221,12cyclictest0-21swapper/010:51:380
2047599221,11cyclictest0-21swapper/010:58:060
2047599220,6cyclictest0-21swapper/011:51:050
111002220,7sleep07-21ksoftirqd/008:00:110
2049399217,3cyclictest86950irq/126-enp1s0-08:15:193
2049399216,7cyclictest86950irq/126-enp1s0-11:05:113
2049399215,9cyclictest14487-21kworker/u8:111:59:273
2049399215,9cyclictest14487-21kworker/u8:111:59:273
2049399215,4cyclictest31094-21chrt09:59:133
2049399214,8cyclictest86950irq/126-enp1s0-11:40:513
2049399213,5cyclictest22086-21kworker/u8:009:42:033
2049399213,2cyclictest16609-21kworker/u8:009:21:073
2049399212,11cyclictest19164-21grep09:30:203
2049399211,6cyclictest0-21swapper/311:51:013
2049399211,6cyclictest0-21swapper/310:46:503
2049399211,6cyclictest0-21swapper/309:46:293
2049399211,4cyclictest0-21swapper/311:01:303
2049399211,2cyclictest0-21swapper/312:22:143
20493992111,3cyclictest87250irq/129-enp1s0-12:05:153
2049399211,12cyclictest2261-21awk12:35:013
2049399211,11cyclictest801-21snmpd08:22:223
20493992110,3cyclictest87250irq/129-enp1s0-07:15:263
20487992114,4cyclictest18186-21grep08:15:152
2048799211,3cyclictest24398-21kworker/u8:109:57:342
2048799211,13cyclictest0-21swapper/208:55:262
20479992117,2cyclictest0-21swapper/110:55:171
2047999211,19cyclictest0-21swapper/107:47:061
2047999211,18cyclictest0-21swapper/109:44:041
2047999211,18cyclictest0-21swapper/109:11:071
2047999210,19cyclictest0-21swapper/109:20:141
2047599215,3cyclictest29755-21perl09:55:160
2047599211,9cyclictest20690-21/usr/sbin/munin07:10:280
2047599211,7cyclictest5019-21/usr/sbin/munin09:00:160
2047599211,6cyclictest23498-21sshd12:08:320
2047599211,4cyclictest0-21swapper/012:01:270
2047599211,3cyclictest87150irq/128-enp1s0-08:50:130
2047599211,3cyclictest86950irq/126-enp1s0-11:11:310
2047599211,11cyclictest0-21swapper/011:26:100
2047599211,10cyclictest0-21swapper/007:44:320
2047599210,9cyclictest11642-21cpu09:15:100
2049399209,3cyclictest5019-21/usr/sbin/munin09:00:223
2049399205,4cyclictest801-21snmpd08:06:083
2049399201,9cyclictest0-21swapper/307:40:113
2049399201,6cyclictest0-21swapper/309:10:093
2049399201,3cyclictest0-21swapper/310:31:133
2049399201,3cyclictest0-21swapper/310:25:053
2049399201,12cyclictest0-21swapper/307:55:203
2049399201,11cyclictest28582-21/usr/sbin/munin08:40:233
2049399201,10cyclictest30167-21/usr/sbin/munin07:30:193
2048799206,5cyclictest23543-21expr10:55:152
2048799201,9cyclictest0-21swapper/212:25:482
2048799201,9cyclictest0-21swapper/210:50:192
2048799201,5cyclictest0-21swapper/212:30:132
2048799201,5cyclictest0-21swapper/210:19:202
20487992012,5cyclictest2130-21grep08:50:262
2048799201,13cyclictest0-21swapper/212:05:192
2047999205,2cyclictest87250irq/129-enp1s0-08:21:321
2047999203,6cyclictest87250irq/129-enp1s0-11:30:121
20479992016,3cyclictest0-21swapper/109:02:081
2047999201,4cyclictest23622-21sh12:10:011
2047999201,3cyclictest27207-21kernelversion08:35:181
2047999201,3cyclictest0-21swapper/110:01:441
2047999201,18cyclictest0-21swapper/107:34:031
2047999201,17cyclictest0-21swapper/108:05:241
2047999201,16cyclictest0-21swapper/111:55:361
2047999201,16cyclictest0-21swapper/111:55:361
2047999201,10cyclictest17911-21chrt09:29:131
2047999201,10cyclictest17911-21chrt09:29:131
2047599208,3cyclictest0-21swapper/011:00:150
2047599201,9cyclictest0-21swapper/007:45:220
2047599201,8cyclictest0-21swapper/012:20:370
2047599201,8cyclictest0-21swapper/010:10:490
2047599201,8cyclictest0-21swapper/009:32:010
2047599201,8cyclictest0-21swapper/009:12:160
2047599201,7cyclictest20411-21/usr/sbin/munin09:35:240
2047599201,2cyclictest0-21swapper/010:26:350
2047599201,16cyclictest23372-21awk09:40:160
2047599201,10cyclictest0-21swapper/011:55:120
2047599201,10cyclictest0-21swapper/011:55:120
2047599200,4cyclictest555-21needreboot11:15:180
257752190,2sleep30-21swapper/307:20:133
2049399199,3cyclictest30441-21/usr/sbin/munin12:25:233
2049399198,3cyclictest32363-21/usr/sbin/munin07:35:203
2049399196,3cyclictest986-1kworker/u9:110:10:153
2049399191,8cyclictest16919-21grep08:10:263
2049399191,6cyclictest7291-21cpuspeed_turbos09:05:133
2049399191,6cyclictest27080-21cat08:35:173
2049399191,6cyclictest26157-21crond08:35:013
2049399191,5cyclictest29944-21perf07:30:013
2049399191,10cyclictest801-21snmpd08:55:183
2049399191,10cyclictest30780-21/usr/sbin/munin08:45:153
2048799197,3cyclictest12050irq/123-ahci[0007:40:012
2048799194,6cyclictest25974-21ps09:45:212
2048799192,2cyclictest0-21swapper/210:48:012
2048799191,8cyclictest9193-21diskmemload09:15:102
2048799191,8cyclictest0-21swapper/211:08:332
2048799191,7cyclictest0-21swapper/212:20:172
2048799191,7cyclictest0-21swapper/210:20:552
2048799191,6cyclictest27916-21kworker/u8:012:35:342
2048799191,6cyclictest0-21swapper/209:23:052
2048799190,8cyclictest0-21swapper/207:25:182
2048799190,3cyclictest22086-21kworker/u8:009:52:362
2047999192,3cyclictest26678-21smart_sda11:00:251
2047999192,14cyclictest25482-21hddtemp_smartct09:45:151
20479991916,2cyclictest0-21swapper/107:21:261
2047999191,4cyclictest932-21wc07:35:191
2047999191,3cyclictest3793-21diskstats10:10:141
2047999191,3cyclictest12576-21diskstats10:30:131
2047999191,3cyclictest10720-21tr11:40:131
2047999191,2cyclictest0-21swapper/107:19:101
2047599197,2cyclictest9935-21grep07:55:220
2047599196,3cyclictest0-21swapper/012:17:120
2047599196,2cyclictest0-21swapper/008:10:390
2047599193,5cyclictest0-21swapper/010:35:580
2047599193,2cyclictest8625-21ls11:35:160
2047599191,6cyclictest16622-21/usr/sbin/munin10:40:120
2047599191,6cyclictest16622-21/usr/sbin/munin10:40:120
2047599191,4cyclictest4446-21sshd08:55:490
2047599191,3cyclictest2689-21cat12:35:140
20475991913,3cyclictest30534-21grep07:30:160
2047599190,8cyclictest0-21swapper/009:50:210
2047599190,10cyclictest0-21swapper/009:45:190
2048799186,3cyclictest86950irq/126-enp1s0-07:10:142
2048799183,6cyclictest30-21rcuc/211:20:232
2048799183,3cyclictest86950irq/126-enp1s0-07:20:232
2048799181,7cyclictest7514-21smart_sda11:30:252
2048799181,7cyclictest24525-21df08:30:132
2048799181,6cyclictest29393-21irqstats08:40:182
2048799181,6cyclictest0-21swapper/211:14:192
2048799181,6cyclictest0-21swapper/210:30:112
2048799181,6cyclictest0-21swapper/209:30:002
2048799181,6cyclictest0-21swapper/209:30:002
2048799181,2cyclictest32-21ksoftirqd/212:13:132
20487991812,3cyclictest11076-21ntpq10:25:212
2048799181,10cyclictest0-21swapper/207:15:242
20479991814,2cyclictest0-21swapper/109:15:221
2047999181,3cyclictest24921-21ntp_states12:10:211
20479991811,4cyclictest19266-21ntp_offset09:30:221
2047599185,2cyclictest835-21runrttasks08:37:250
2047599184,3cyclictest0-21swapper/008:28:230
2047599183,3cyclictest13813-21/usr/sbin/munin09:20:210
2047599181,8cyclictest0-21swapper/010:06:390
20475991813,3cyclictest24997-21tr12:10:220
20475991812,3cyclictest20385-21gltestperf08:20:160
2047599181,15cyclictest0-21swapper/008:05:130
2047599180,9cyclictest0-21swapper/007:25:210
2047599180,7cyclictest0-21swapper/008:40:090
169862181,2sleep20-21swapper/210:40:142
169862181,2sleep20-21swapper/210:40:142
2049399171,9cyclictest0-21swapper/307:10:223
2049399171,13cyclictest4398-21/usr/sbin/munin07:45:173
2049399170,4cyclictest1013-21fschecks_count08:50:163
2048799176,3cyclictest86950irq/126-enp1s0-07:55:132
2048799175,2cyclictest86950irq/126-enp1s0-11:40:382
2048799174,5cyclictest30-21rcuc/208:35:232
2048799173,5cyclictest32-21ksoftirqd/211:47:522
2048799173,2cyclictest86950irq/126-enp1s0-08:07:312
2048799172,6cyclictest30780-21/usr/sbin/munin08:45:242
2048799171,7cyclictest9193-21diskmemload11:50:162
2048799171,7cyclictest4785-21fschecks_count07:45:162
2048799171,7cyclictest0-21swapper/211:15:052
2048799171,7cyclictest0-21swapper/210:39:342
2048799171,7cyclictest0-21swapper/210:13:122
20487991713,3cyclictest0-21swapper/208:26:562
20487991711,4cyclictest20505-21sort08:20:172
2048799171,14cyclictest0-21swapper/207:40:032
20487991711,3cyclictest11166-21cat08:00:112
2048799171,10cyclictest0-21swapper/209:05:172
2048799171,10cyclictest0-21swapper/208:12:262
2047599171,6cyclictest0-21swapper/009:05:240
20475991711,3cyclictest23584-21acpi07:15:100
20475991710,4cyclictest6742-21tr07:50:110
324642160,2sleep00-21swapper/008:46:310
2049399161,4cyclictest23209-21processes08:25:243
2048799168,5cyclictest786-21sssd_be09:00:152
2048799161,6cyclictest30167-21/usr/sbin/munin07:30:182
2048799161,5cyclictest0-21swapper/207:50:142
20475991611,3cyclictest32714-21dnf07:35:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional