You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 06:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Dec 23, 2025 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
132372217139,66sleep00-21swapper/019:09:500
131152216165,39sleep20-21swapper/219:08:152
131162202166,24sleep30-21swapper/319:08:173
130562200163,24sleep10-21swapper/119:07:291
1293121760,4sleep21334699cyclictest00:00:192
245472660,4sleep339-21rcuc/319:30:263
255052550,3sleep01333699cyclictest22:00:250
14072520,4sleep19-21rcu_preempt19:55:001
13346995248,3cyclictest0-21swapper/223:13:012
122122460,1sleep081ktimersoftd/022:45:190
1333699382,25cyclictest28797-21kworker/0:323:49:110
255742360,1sleep20-21swapper/223:15:192
1333699342,25cyclictest28643-21kworker/0:122:18:390
1333699333,23cyclictest8559-21kworker/0:121:27:210
1333699332,24cyclictest11355-21kworker/0:122:56:080
1333699331,8cyclictest29800-21kworker/0:221:08:180
1333699331,14cyclictest3529-21kworker/0:020:00:230
1333699324,13cyclictest28433-21munin-run22:10:000
1333699322,24cyclictest2012-21kworker/0:121:15:420
1333699322,24cyclictest2012-21kworker/0:121:15:420
1333699312,24cyclictest18208-21kworker/0:119:44:270
1333699304,13cyclictest4873-21idleruntime-cro22:30:010
1333699301,20cyclictest11764-21kworker/0:200:00:490
1335199291,7cyclictest0-21swapper/321:32:153
1334199293,17cyclictest904-21sssd_nss21:10:201
1334199293,14cyclictest11694-40dnf23:55:341
1333699295,11cyclictest22463-21awk23:10:020
1333699292,25cyclictest25124-21kworker/0:200:36:530
1333699292,10cyclictest14965-21kworker/0:021:57:570
1333699291,23cyclictest29228-21kworker/0:019:50:330
1333699291,10cyclictest2496-21kworker/0:322:40:210
1333699290,18cyclictest28797-21kworker/0:323:23:370
1333699282,10cyclictest5792-21kworker/0:220:07:310
1333699282,10cyclictest5792-21kworker/0:220:07:310
1333699282,10cyclictest11764-21kworker/0:223:58:490
1333699272,10cyclictest28643-21kworker/0:122:13:530
1333699271,24cyclictest16245-21kworker/0:300:13:490
1333699271,10cyclictest9116-21kworker/0:022:38:490
13341992615,3cyclictest3684-21/usr/sbin/munin22:25:191
13341992612,12cyclictest13511-21kworker/u8:122:58:321
1334199261,19cyclictest0-21swapper/100:37:311
1333699262,9cyclictest5792-21kworker/0:220:37:570
1333699262,9cyclictest29228-21kworker/0:019:48:570
1333699262,16cyclictest19934-21kworker/0:223:11:260
1333699262,10cyclictest25124-21kworker/0:200:34:550
1333699262,10cyclictest2012-21kworker/0:121:14:100
1333699261,17cyclictest28797-21kworker/0:323:36:130
1333699261,12cyclictest14617-21munin-run20:25:010
1333699261,10cyclictest17374-21kworker/0:121:46:590
1333699261,10cyclictest15276-21kworker/0:219:13:200
19142250,3sleep339-21rcuc/319:55:163
1334199251,19cyclictest0-21swapper/119:25:241
1334199251,16cyclictest0-21swapper/123:20:221
1334199251,16cyclictest0-21swapper/121:56:571
1333699252,9cyclictest11764-21kworker/0:200:05:350
1333699252,17cyclictest8559-21kworker/0:121:31:370
1333699252,10cyclictest6326-21kworker/0:021:23:490
1333699251,11cyclictest28797-21kworker/0:323:40:390
1333699251,11cyclictest28797-21kworker/0:323:40:390
1333699251,10cyclictest24585-21kworker/0:123:15:060
1333699251,10cyclictest10805-21kworker/0:221:37:270
13336992510,3cyclictest7312-21kworker/0:123:50:170
1335199241,5cyclictest0-21swapper/322:36:213
1335199241,12cyclictest0-21swapper/321:58:013
13341992411,4cyclictest1612-21df23:35:141
1333699248,3cyclictest2496-21kworker/0:322:30:190
1333699248,3cyclictest2496-21kworker/0:322:30:190
1333699242,9cyclictest14965-21kworker/0:021:43:330
1333699242,13cyclictest19934-21kworker/0:223:28:250
1333699241,13cyclictest5792-21kworker/0:220:30:110
1333699241,10cyclictest16548-21kworker/0:020:29:090
1333699241,10cyclictest14965-21kworker/0:021:54:270
260092231,3sleep126167-21df_inode20:50:161
1335199236,7cyclictest0-21swapper/322:42:353
1335199231,17cyclictest0-21swapper/300:34:363
1335199231,11cyclictest801-21snmpd21:25:133
1334199231,3cyclictest14489-21sshd20:23:321
1334199231,16cyclictest0-21swapper/119:19:251
1333699239,2cyclictest16245-21kworker/0:300:19:030
1333699237,3cyclictest32036-21kworker/0:021:03:520
1333699235,2cyclictest3529-21kworker/0:019:59:150
1333699232,9cyclictest11355-21kworker/0:122:51:260
1333699231,12cyclictest25267-21kworker/0:020:49:130
1333699231,10cyclictest28797-21kworker/0:323:30:270
151062220,2sleep230-21rcuc/220:25:112
1335199227,9cyclictest0-21swapper/323:34:273
1335199227,4cyclictest0-21swapper/321:35:023
1335199226,6cyclictest0-21swapper/322:27:523
1335199222,11cyclictest5693-21/usr/sbin/munin23:45:123
1334199222,17cyclictest20600-21munin-run19:25:011
13341992216,3cyclictest24738-21timerwakeupswit00:25:261
13341992214,2cyclictest0-21swapper/123:43:071
13341992214,2cyclictest0-21swapper/123:43:061
1334199221,16cyclictest0-21swapper/122:52:421
1334199221,16cyclictest0-21swapper/121:48:361
1334199221,15cyclictest0-21swapper/121:41:271
1334199221,10cyclictest0-21swapper/122:36:581
1334199221,10cyclictest0-21swapper/100:18:281
1333699222,9cyclictest20562-21kworker/0:000:20:110
1333699222,9cyclictest11355-21kworker/0:123:04:420
1333699222,10cyclictest32621-21kworker/0:022:23:010
1333699220,7cyclictest5792-21kworker/0:220:10:290
1335199217,3cyclictest0-21swapper/300:10:533
1335199216,4cyclictest0-21swapper/300:06:193
1335199214,7cyclictest0-21swapper/322:30:373
1335199214,7cyclictest0-21swapper/322:30:373
1335199211,11cyclictest2052-21diskmemload22:45:293
1335199210,11cyclictest6183-21kworker/u8:221:22:373
1334699211,9cyclictest0-21swapper/200:10:302
13346992115,3cyclictest2637-21strings19:55:212
1334699211,13cyclictest0-21swapper/220:05:272
1334699211,13cyclictest0-21swapper/220:05:272
1334699211,12cyclictest0-21swapper/221:45:182
1334699211,11cyclictest20473-21grep00:15:282
1334199219,3cyclictest0-21swapper/122:15:531
1334199218,3cyclictest0-21swapper/122:10:001
1334199211,17cyclictest15233-21munin-run21:40:021
1334199211,15cyclictest0-21swapper/122:10:391
1334199211,15cyclictest0-21swapper/120:27:221
1334199211,15cyclictest0-21swapper/119:46:311
1334199211,14cyclictest0-21swapper/119:40:441
1334199211,14cyclictest0-21swapper/100:06:161
1334199211,11cyclictest7703-21cat23:50:011
1334199211,11cyclictest21885-21fschecks_count20:40:151
1334199211,10cyclictest0-21swapper/122:33:531
1334199211,10cyclictest0-21swapper/122:33:531
1334199211,10cyclictest0-21swapper/100:22:031
1334199211,10cyclictest0-21swapper/100:02:511
1334199210,4cyclictest18576-21kworker/u8:100:31:031
1334199210,13cyclictest7425-21kworker/u8:223:51:511
1334199210,12cyclictest0-21swapper/123:05:281
1333699218,3cyclictest5792-21kworker/0:220:44:590
1333699218,3cyclictest10761-21kworker/0:019:16:270
1333699217,2cyclictest18208-21kworker/0:119:37:090
1333699213,2cyclictest25216-21kworker/0:300:29:390
1333699213,2cyclictest20518-21kworker/0:319:26:110
1333699213,10cyclictest18629-21/usr/sbin/munin19:20:190
1335199208,4cyclictest0-21swapper/300:20:103
1335199207,3cyclictest0-21swapper/300:36:303
1335199206,4cyclictest14992-21tr20:25:123
1335199206,3cyclictest30749-21sed23:25:243
1335199205,4cyclictest0-21swapper/323:41:283
1335199205,4cyclictest0-21swapper/323:41:283
1335199205,3cyclictest86950irq/126-enp1s0-22:51:513
1335199204,6cyclictest0-21swapper/300:04:253
1335199204,3cyclictest0-21swapper/323:37:093
1335199201,12cyclictest0-21swapper/321:18:403
1335199201,12cyclictest0-21swapper/321:18:403
1335199201,11cyclictest18919-21/usr/sbin/munin00:15:263
1334699209,2cyclictest29682-21awk00:40:002
1334699206,5cyclictest0-21swapper/222:02:122
1334699202,3cyclictest0-21swapper/221:28:452
1334699201,12cyclictest0-21swapper/222:30:172
1334699201,12cyclictest0-21swapper/222:30:172
13341992012,3cyclictest0-21swapper/120:39:291
13341992012,3cyclictest0-21swapper/119:33:111
13341992012,2cyclictest0-21swapper/121:31:561
1334199201,18cyclictest0-21swapper/120:55:191
1334199201,17cyclictest0-21swapper/122:40:181
1334199201,14cyclictest0-21swapper/123:33:051
1334199201,14cyclictest0-21swapper/121:21:431
1334199201,14cyclictest0-21swapper/121:18:111
1334199201,14cyclictest0-21swapper/121:18:111
1334199201,14cyclictest0-21swapper/121:09:161
1334199201,14cyclictest0-21swapper/121:03:041
1334199201,14cyclictest0-21swapper/120:05:441
1334199201,14cyclictest0-21swapper/120:05:441
1334199201,14cyclictest0-21swapper/119:35:471
1334199201,11cyclictest23353-21hddtemp_smartct23:10:171
1334199201,11cyclictest0-21swapper/123:29:481
1334199201,11cyclictest0-21swapper/121:25:181
1334199201,10cyclictest1703-21ntpq22:20:221
1334199200,15cyclictest0-21swapper/120:46:121
1334199200,14cyclictest0-21swapper/120:31:371
1333699207,3cyclictest25267-21kworker/0:020:54:520
208972191,3sleep120918-21ntp_states21:50:231
200952191,15sleep10-21swapper/123:03:071
1335199198,3cyclictest28654-21/usr/sbin/munin22:10:233
1335199198,3cyclictest16431-21/usr/sbin/munin19:15:253
1335199197,4cyclictest0-21swapper/323:55:373
1335199197,3cyclictest0-21swapper/322:01:243
1335199195,7cyclictest13511-21kworker/u8:123:13:013
1335199194,9cyclictest0-21swapper/321:45:263
1335199193,9cyclictest0-21swapper/322:57:503
1335199193,7cyclictest0-21swapper/322:23:013
1335199193,3cyclictest0-21swapper/321:40:583
13351991913,3cyclictest26424-21grep23:15:283
1335199191,12cyclictest0-21swapper/323:06:173
1335199191,11cyclictest16494-21kworker/u8:221:03:133
1335199191,11cyclictest0-21swapper/322:16:083
1335199191,10cyclictest28374-21chrt22:09:313
1334699196,4cyclictest13978-21basename21:35:162
1334699192,6cyclictest15657-21munin-node20:15:012
1334699191,8cyclictest7233-21unixbench_singl23:45:262
1334699191,6cyclictest593-21/usr/sbin/munin22:20:192
1334699191,5cyclictest0-21swapper/200:06:262
1334699191,2cyclictest87150irq/128-enp1s0-23:55:222
1334199197,2cyclictest30382-21kworker/u8:119:11:351
1334199195,4cyclictest0-21swapper/122:47:291
1334199191,9cyclictest0-21swapper/100:13:591
1334199191,10cyclictest0-21swapper/123:15:151
1334199190,14cyclictest0-21swapper/120:16:591
1334199190,14cyclictest0-21swapper/120:11:451
1334199190,14cyclictest0-21swapper/120:01:501
1333699191,11cyclictest5792-21kworker/0:220:18:230
1335199184,4cyclictest18525-21unixbench-2d20:30:263
1335199181,9cyclictest27069-21/usr/sbin/munin23:20:183
1335199181,9cyclictest0-21swapper/321:14:263
1335199181,5cyclictest0-21swapper/323:53:263
13351991813,3cyclictest23679-21grep00:25:163
13351991812,3cyclictest27794-21idleruntime-cro20:55:003
1335199181,15cyclictest0-21swapper/320:37:123
13351991811,4cyclictest22057-21find19:25:233
1335199181,11cyclictest0-21swapper/323:02:563
1335199181,11cyclictest0-21swapper/321:50:213
1335199181,10cyclictest3843-21/usr/sbin/munin20:00:203
1334699186,4cyclictest7301-21gltestperf21:20:152
1334699181,6cyclictest5469-21munin-run23:45:022
1334699181,6cyclictest5469-21munin-run23:45:022
1334699181,6cyclictest19037-21if_enp2s023:00:182
1334699181,6cyclictest0-21swapper/222:13:392
1334699181,6cyclictest0-21swapper/200:25:502
1334699181,5cyclictest18666-21kworker/2:020:38:412
1334699181,5cyclictest0-21swapper/222:05:442
1334699181,4cyclictest0-21swapper/221:14:182
1334699181,4cyclictest0-21swapper/200:31:092
1334699181,3cyclictest0-21swapper/223:35:522
1334699181,3cyclictest0-21swapper/221:41:052
1334699181,3cyclictest0-21swapper/221:01:052
13346991813,3cyclictest22659-21grep00:20:292
13346991813,3cyclictest22449-21munin-plugin-st23:10:012
1334199185,4cyclictest24669-21sort22:00:161
1334199181,2cyclictest0-21swapper/119:56:071
1333699182,10cyclictest27496-21kworker/0:120:56:260
13336991810,5cyclictest24532-21grep19:30:260
54572171,2sleep20-21swapper/220:00:362
255652170,3sleep30-21swapper/320:49:453
253902171,2sleep20-21swapper/220:47:322
233102171,3sleep20-21swapper/220:43:592
1335199178,3cyclictest25143-21kworker/u8:019:35:543
1335199171,9cyclictest0-21swapper/319:44:313
1335199171,3cyclictest87250irq/129-enp1s0-20:10:173
13351991712,3cyclictest13537-21/usr/sbin/munin19:10:263
1335199171,15cyclictest0-21swapper/319:46:453
13351991711,3cyclictest801-21snmpd20:09:593
13351991711,3cyclictest801-21snmpd20:09:593
13351991711,3cyclictest28029-21/usr/sbin/munin20:55:193
13351991711,3cyclictest22359-21sort20:40:203
1334699172,7cyclictest0-21swapper/222:28:112
1334699172,7cyclictest0-21swapper/221:18:322
1334699172,7cyclictest0-21swapper/221:18:322
1334699172,5cyclictest577-21systemd-journal23:50:292
1334699172,5cyclictest20438-21/usr/sbin/munin21:50:192
1334699171,7cyclictest0-21swapper/223:23:152
1334699171,7cyclictest0-21swapper/222:49:432
1334699171,6cyclictest0-21swapper/222:56:102
1334699171,5cyclictest18666-21kworker/2:021:33:372
1334699171,3cyclictest0-21swapper/222:50:442
13346991713,2cyclictest2052-21diskmemload23:27:382
13346991712,3cyclictest27995-21cut20:55:012
1334699171,15cyclictest0-21swapper/219:43:272
13346991711,3cyclictest7543-21expr22:35:112
13346991711,3cyclictest16357-21expr19:15:042
13346991711,3cyclictest14124-21grep19:10:152
13346991710,4cyclictest28619-21grep20:55:172
11222170,4sleep21334699cyclictest21:05:212
236862161,2sleep20-21swapper/221:55:552
1335199163,4cyclictest11298-21sort20:15:183
1335199161,4cyclictest0-21swapper/320:22:263
13351991612,3cyclictest0-21swapper/319:23:503
13351991610,3cyclictest31805-21/usr/sbin/munin19:50:153
1334699169,4cyclictest9583-21grep22:40:132
1334699169,4cyclictest24393-21cat19:30:272
1334699163,4cyclictest11607-21ps20:15:222
1334699162,6cyclictest0-21swapper/219:53:082
1334699161,9cyclictest0-21swapper/219:20:462
1334699161,5cyclictest0-21swapper/222:17:192
1334699161,5cyclictest0-21swapper/219:25:162
13346991610,3cyclictest31586-21expr23:30:132
13351991510,3cyclictest32502-21wc21:05:113
1334699154,4cyclictest13452-21interrupts20:20:172
1334699152,10cyclictest30758-21tr19:45:222
13346991510,3cyclictest17210-21cstates20:30:122
1334699149,3cyclictest25506-21expr19:35:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional