You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 06:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Dec 10, 2025 00:43:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2413893212852,71sleep30-21swapper/319:06:073
241401528959,10sleep20-21swapper/219:07:382
241403728858,9sleep10-21swapper/119:07:521
241403628061,14sleep00-21swapper/019:07:510
25845342380,2sleep10-21swapper/121:33:181
2414335993834,2cyclictest0-21swapper/300:15:013
26961312330,2sleep20-21swapper/222:39:522
2414335992926,1cyclictest0-21swapper/322:40:003
2414335992924,3cyclictest0-21swapper/321:35:213
2414335992924,2cyclictest0-21swapper/323:10:273
2414335992724,1cyclictest0-21swapper/322:55:013
2414335992723,2cyclictest0-21swapper/321:20:523
2414335992722,2cyclictest66550irq/131-enp1s0-TxRx-322:40:523
241433599269,12cyclictest2636026-21taskset22:03:493
2414335992623,1cyclictest0-21swapper/300:32:453
2414335992622,2cyclictest0-21swapper/322:56:243
2414335992622,2cyclictest0-21swapper/321:49:363
2414335992622,2cyclictest0-21swapper/321:49:363
2414335992522,2cyclictest0-21swapper/323:35:013
2414335992521,2cyclictest0-21swapper/321:17:593
2414335992520,2cyclictest0-21swapper/322:30:183
2414335992421,1cyclictest0-21swapper/322:10:203
2414335992421,1cyclictest0-21swapper/319:15:003
2414335992420,2cyclictest0-21swapper/323:40:183
241433599241,19cyclictest2895563-21taskset00:36:393
26455902231,15sleep22645621-21ssh22:10:112
25837312231,2sleep00-21swapper/021:32:090
2414335992320,1cyclictest0-21swapper/323:27:573
2414335992319,2cyclictest0-21swapper/323:50:203
2414335992317,2cyclictest0-21swapper/323:47:203
241433599222,15cyclictest2846201-21taskset00:05:243
2414335992219,1cyclictest0-21swapper/319:40:193
2414335992218,2cyclictest0-21swapper/323:17:193
2414335992218,2cyclictest0-21swapper/323:09:343
2414335992218,2cyclictest0-21swapper/321:30:213
2414335992217,2cyclictest0-21swapper/300:15:383
241433599221,15cyclictest2656291-21chrt22:15:163
28495352211,5sleep11-21systemd00:10:001
2414335992118,2cyclictest0-21swapper/321:55:013
2414335992118,2cyclictest0-21swapper/321:55:003
2414335992118,1cyclictest0-21swapper/322:48:143
2414335992118,1cyclictest0-21swapper/322:48:133
2414335992118,1cyclictest0-21swapper/320:45:003
2414335992117,2cyclictest0-21swapper/323:20:213
2414335992117,2cyclictest0-21swapper/322:10:003
2414335992115,3cyclictest0-21swapper/300:00:183
241433599211,14cyclictest2543130-21turbostat21:10:013
26763582201,3sleep10-21swapper/122:25:571
26105972201,3sleep12543122-21diskmemload21:48:581
26105972201,3sleep12543122-21diskmemload21:48:571
25564992201,5sleep12556560-21ntpq21:15:221
241433599208,7cyclictest2797225-21taskset23:37:383
241433599205,12cyclictest0-21swapper/321:57:153
2414335992018,1cyclictest0-21swapper/321:40:023
2414335992017,1cyclictest0-21swapper/322:20:223
2414335992017,1cyclictest0-21swapper/321:25:253
2414335992016,2cyclictest0-21swapper/321:10:223
2414335992015,2cyclictest0-21swapper/300:24:233
241433599201,16cyclictest0-21swapper/319:20:113
241433599200,17cyclictest66550irq/131-enp1s0-TxRx-323:56:273
241433599194,13cyclictest0-21swapper/320:10:173
2414335991916,1cyclictest0-21swapper/322:25:253
2414335991916,1cyclictest0-21swapper/321:00:253
2414335991916,1cyclictest0-21swapper/320:15:253
2414335991914,2cyclictest0-21swapper/300:25:563
241433599191,15cyclictest0-21swapper/319:20:013
28813542181,1sleep00-21swapper/000:29:360
28722782181,9sleep12872285-21ssh00:22:351
27122862181,2sleep10-21swapper/122:48:181
27122862181,2sleep10-21swapper/122:48:181
24486912181,7sleep02448925-21kthreadcore19:40:150
2414335991816,1cyclictest0-21swapper/320:45:203
2414335991816,1cyclictest0-21swapper/320:05:193
2414335991815,1cyclictest0-21swapper/320:40:003
2414335991815,1cyclictest0-21swapper/319:35:183
2414335991814,2cyclictest0-21swapper/320:05:003
2414335991814,2cyclictest0-21swapper/319:50:183
2414335991814,2cyclictest0-21swapper/319:30:143
2414335991814,2cyclictest0-21swapper/319:30:013
2414335991813,2cyclictest0-21swapper/323:00:133
241433599181,13cyclictest0-21swapper/319:55:123
28570222171,3sleep00-21swapper/000:14:160
28337552171,3sleep22833785-21apt-get00:00:122
28311942171,10sleep12543122-21diskmemload23:56:531
27872622171,4sleep22543122-21diskmemload23:30:492
27543492171,3sleep20-21swapper/223:12:452
27048292171,10sleep00-21swapper/022:44:570
27029052171,4sleep12543122-21diskmemload22:42:221
27024532171,4sleep22543122-21diskmemload22:41:462
26090532171,4sleep22543122-21diskmemload21:46:472
26090532171,4sleep22543122-21diskmemload21:46:462
25913572171,4sleep22543122-21diskmemload21:35:592
2414335991715,1cyclictest0-21swapper/320:21:203
2414335991714,1cyclictest0-21swapper/320:25:203
2414335991713,2cyclictest0-21swapper/320:55:103
2414335991711,3cyclictest0-21swapper/320:50:143
241433599170,11cyclictest2505719-21chrt20:32:463
28654082161,3sleep00-21swapper/000:19:290
27205732161,2sleep00-21swapper/022:53:020
27190022161,4sleep12543122-21diskmemload22:50:551
26952032161,2sleep00-21swapper/022:38:310
26687122161,3sleep12543122-21diskmemload22:22:171
26365012161,3sleep00-21swapper/022:04:320
2414335991614,1cyclictest0-21swapper/319:45:183
28957932151,8sleep12543122-21diskmemload00:37:061
28143252151,3sleep12543122-21diskmemload23:47:231
28137612151,2sleep00-21swapper/023:46:390
28035682150,2sleep20-21swapper/223:40:242
27779822151,3sleep12778005-21switchtime23:25:271
26890032151,9sleep12690384-21cut22:35:191
26860332151,7sleep20-21swapper/222:32:122
25948282150,1sleep20-21swapper/221:40:092
25852942150,1sleep20-21swapper/221:34:232
25756212151,10sleep22575634-21sh21:27:332
25441382151,9sleep22545605-21ssh21:10:182
28744862141,9sleep12874544-21expr00:25:161
28069092141,9sleep02806921-21ssh23:44:060
27960252141,7sleep22543122-21diskmemload23:35:582
27908032143,6sleep12790830-21apt-get23:35:131
27662132141,9sleep22767799-21grep23:20:182
27366342141,9sleep02736648-21ssh23:01:400
27208912141,2sleep244-21ksoftirqd/222:53:282
26769102141,2sleep20-21swapper/222:26:422
26621542141,8sleep12543122-21diskmemload22:20:001
26264092141,9sleep02626422-21ssh21:57:020
25596212141,5sleep20-21swapper/221:18:472
25495102141,2sleep00-21swapper/021:12:050
24583282141,9sleep02458331-21sshd19:48:390
24381822141,8sleep02439513-21cat19:30:160
24217492140,3sleep00-21swapper/019:15:130
28974652131,4sleep02897480-21ssh00:39:510
28885742131,8sleep12888587-21ssh00:33:291
28882372132,7sleep2802-21snmpd00:32:592
28731682131,8sleep22873178-21ssh00:24:012
28470642131,8sleep02847067-21ssh00:05:550
27806312131,7sleep02543122-21diskmemload23:28:350
27614172131,2sleep10-21swapper/123:15:481
27351522131,8sleep22735217-21/usr/sbin/munin23:00:232
26508102131,9sleep12650823-21rm22:10:591
26172332131,8sleep22617243-21ssh21:51:002
26172332131,8sleep22617243-21ssh21:50:592
26021912131,7sleep02602205-21ssh21:44:000
25515952131,7sleep12551609-21ssh21:14:581
24745002133,4sleep22474519-21apt-get20:05:132
24337992131,3sleep22435575-21kthreadcore19:25:182
28825472121,7sleep02883145-21cut00:30:160
28803452121,4sleep20-21swapper/200:27:492
28646852120,8sleep12864696-21ssh00:18:261
28577462121,7sleep22543122-21diskmemload00:15:022
28558112121,7sleep22543122-21diskmemload00:12:192
28504712121,7sleep12850705-21kthreadcore00:10:171
28303692121,5sleep22543122-21diskmemload23:55:462
28228932121,4sleep22543122-21diskmemload23:52:142
28171272121,7sleep02817292-21/usr/sbin/munin23:50:150
28060162121,6sleep12543122-21diskmemload23:42:541
27455392120,1sleep00-21swapper/023:07:180
27374672121,7sleep12543122-21diskmemload23:02:491
27295022121,4sleep12543122-21diskmemload22:58:281
27125472121,7sleep22712560-21rm22:48:392
27125472121,7sleep22712560-21rm22:48:392
26771802121,5sleep00-21swapper/022:27:040
26547232121,4sleep20-21swapper/222:15:162
26338862121,5sleep10-21swapper/122:00:451
26008172121,4sleep12543122-21diskmemload21:42:011
25664262120,7sleep12543122-21diskmemload21:21:241
25582752121,7sleep02558288-21rm21:16:570
28954412111,4sleep22543122-21diskmemload00:36:272
28705692111,6sleep02870584-21sendmail_mailtr00:20:260
28328342111,3sleep00-21swapper/023:59:180
28225752111,4sleep12543122-21diskmemload23:51:481
28143932111,3sleep20-21swapper/223:47:282
27959082111,3sleep00-21swapper/023:35:490
27898992110,1sleep00-21swapper/023:34:190
27877082111,4sleep12543122-21diskmemload23:31:241
27794372111,3sleep20-21swapper/223:27:002
27634362111,3sleep02543122-21diskmemload23:18:370
27539262112,1sleep1321rcuc/123:12:071
27472422111,6sleep22747247-21ssh23:09:402
27460372111,4sleep12543122-21diskmemload23:07:591
27280512111,3sleep20-21swapper/222:56:252
26875912110,6sleep12543122-21diskmemload22:34:271
26701982111,6sleep22670210-21sh22:24:182
26684312110,5sleep00-21swapper/022:21:520
26599092111,4sleep02543122-21diskmemload22:16:500
26372302110,4sleep12637262-21ssh22:05:131
26256442111,4sleep12543122-21diskmemload21:55:551
26181552111,4sleep12543122-21diskmemload21:52:141
26181552111,4sleep12543122-21diskmemload21:52:141
26087562111,3sleep00-21swapper/021:46:200
26087562111,3sleep00-21swapper/021:46:200
25709772111,6sleep12572711-21kthreadcore21:25:201
25638432111,5sleep02564886-21memory21:20:190
25239122110,3sleep02525718-21cut20:50:200
25049042111,5sleep02504950-21ntpq20:30:200
24850912111,5sleep02485343-21df20:15:170
28420022101,3sleep20-21swapper/200:05:122
28406852101,4sleep12543122-21diskmemload00:03:271
28356922101,4sleep02837380-21ssh00:00:170
27702862101,3sleep12543122-21diskmemload23:21:171
27700472101,3sleep00-21swapper/023:20:560
27604942101,3sleep22760545-21ntpq23:15:212
27266652101,4sleep02726725-21ntpq22:55:240
27103962101,3sleep00-21swapper/022:45:500
27103962101,3sleep00-21swapper/022:45:490
26850322101,2sleep00-21swapper/022:30:460
26526432101,3sleep02543122-21diskmemload22:13:350
26448302101,3sleep20-21swapper/222:09:272
26341912101,5sleep22634202-21ssh22:01:102
26281192101,2sleep20-21swapper/221:59:322
26176982101,3sleep00-21swapper/021:51:370
26176982101,3sleep00-21swapper/021:51:370
25857422101,4sleep02543122-21diskmemload21:35:010
25753772101,3sleep00-21swapper/021:27:110
25665272101,3sleep20-21swapper/221:21:322
25421102101,4sleep22542159-21ntpq21:05:222
25278742101,4sleep02528083-21hddtemp_smartct20:55:170
25220482101,4sleep02522073-21munin-run20:50:000
25214342101,4sleep12521678-21users20:45:291
25210052101,4sleep22521053-21ntpq20:45:222
25118112101,4sleep02512027-21diskstats20:40:140
25104392100,2sleep20-21swapper/220:35:202
25102972101,4sleep12510438-21ntpq20:35:181
24953252100,6sleep22495339-21latency_hist20:25:012
24853442101,6sleep12485374-21fschecks_count20:15:181
24639442101,4sleep22463963-21apt-get19:55:112
24574152101,4sleep12457448-21ntpq19:45:201
24185572101,4sleep12418602-21ntpq19:10:211
24150582101,5sleep02415883-21cut19:10:160
2753059290,3sleep00-21swapper/023:10:540
2642094290,5sleep02543122-21diskmemload22:05:320
2587241291,4sleep12588883-21grep21:35:181
2527606291,4sleep22527715-21tr20:55:162
2473909290,3sleep272-21kcompactd020:00:522
2459691291,4sleep12461417-21cut19:50:171
2459242291,4sleep02459690-21grep19:50:170
2432882291,4sleep12433798-21cut19:25:161
2516002280,1sleep20-21swapper/220:40:242
2480054281,3sleep12480151-21fschecks_time20:10:171
2478819281,3sleep02478973-21sed20:05:270
2446909280,4sleep02446960-21ld19:35:210
2430796280,1sleep00-21swapper/019:20:180
2419851281,3sleep12420919-21cut19:15:021
2419454280,1sleep20-21swapper/219:14:152
2542312270,4sleep10-21swapper/121:05:241
2537725270,1sleep00-21swapper/021:03:340
2536929271,3sleep12536985-21cut21:00:231
2516580271,1sleep10-21swapper/120:42:351
2500429270,1sleep10-21swapper/120:27:521
2481037270,3sleep02482783-21cut20:10:210
2469230271,1sleep00-21swapper/020:00:150
2468669270,1sleep10-21swapper/119:56:341
2458191270,1sleep20-21swapper/219:46:532
2448197271,2sleep10-21swapper/119:40:111
2442370270,1sleep10-21swapper/119:31:491
2426752270,1sleep20-21swapper/219:19:572
2542996260,1sleep00-21swapper/021:08:350
2532339260,1sleep10-21swapper/120:57:061
2527131260,3sleep20-21swapper/220:52:382
2506707261,2sleep00-21swapper/020:35:130
2505846260,4sleep10-21swapper/120:34:251
2500503260,1sleep20-21swapper/220:28:502
2496580261,2sleep00-21swapper/020:25:190
2495284260,1sleep10-21swapper/120:24:321
2494983260,1sleep00-21swapper/020:20:430
2484772260,1sleep20-21swapper/220:14:392
2478636260,3sleep10-21swapper/120:05:241
2470381261,2sleep10-21swapper/120:00:191
2453104260,1sleep20-21swapper/219:43:572
2447595260,1sleep20-21swapper/219:36:152
2444945261,2sleep10-21swapper/119:35:181
2441735261,2sleep20-21swapper/219:30:222
2432316260,3sleep00-21swapper/019:25:110
2431053261,2sleep20-21swapper/219:20:202
2533201250,2sleep20-21swapper/221:00:162
2505006251,1sleep20-21swapper/220:30:222
2464122250,1sleep00-21swapper/019:55:130
2463387250,3sleep20-21swapper/219:51:182
2427043250,1sleep10-21swapper/119:20:111
2522089240,1sleep10-21swapper/120:50:011
2485378240,1sleep20-21swapper/220:15:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional