You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 16:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Thu Dec 11, 2025 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24828211083,9sleep30-21swapper/307:06:543
24878210579,9sleep10-21swapper/107:07:341
2472228672,9sleep00-21swapper/007:05:310
2490928265,13sleep20-21swapper/207:07:592
2514099310,2cyclictest224-21systemd-journal12:13:460
2514099310,1cyclictest0-21swapper/012:20:130
2514099281,2cyclictest30495-21apt-get11:30:140
2514099281,24cyclictest31746-21munin-run11:05:000
2514099280,3cyclictest8948-21grep10:20:220
2514099280,3cyclictest29374-21ntpq10:10:200
2514099280,3cyclictest13181-21grep10:50:130
2514099271,2cyclictest3590-21cstates12:00:160
2514099271,25cyclictest613-21aten2_r7power_v10:40:140
2514099270,3cyclictest7744-21snmpd10:26:540
2514099270,2cyclictest7744-21snmpd09:27:440
2514099270,2cyclictest7744-21snmpd09:27:440
2514099270,15cyclictest0-21swapper/010:32:540
25143992625,1cyclictest0-21swapper/112:32:361
25143992624,1cyclictest0-21swapper/112:20:131
2514099262,23cyclictest130952sleep011:15:400
2514099262,23cyclictest130952sleep011:15:400
2514099260,3cyclictest7744-21snmpd09:38:200
2514099260,2cyclictest5476-21taskset10:17:360
2514099260,2cyclictest1291-21runrttasks12:08:260
2514099260,25cyclictest0-21swapper/011:50:150
2514099252,17cyclictest18226-21diskmemload10:45:230
2514099250,2cyclictest30808-21cron09:20:000
2514099250,2cyclictest1291-21runrttasks10:58:440
2514099250,24cyclictest0-21swapper/012:15:170
26782240,1sleep220159-21kworker/2:109:50:052
25143992421,2cyclictest462-21ssh10:40:131
25143992420,3cyclictest0-21swapper/111:05:001
2514099243,18cyclictest18226-21diskmemload11:10:510
2514099241,19cyclictest18226-21diskmemload10:02:220
25143992319,3cyclictest1781-21aten2_r7power_p10:15:141
2514099233,19cyclictest18226-21diskmemload10:06:470
2514099232,20cyclictest18226-21diskmemload11:07:120
2514099231,21cyclictest18226-21diskmemload09:41:460
2514099231,17cyclictest18226-21diskmemload09:45:590
2514099230,20cyclictest0-21swapper/010:35:200
2514099230,1cyclictest0-21swapper/009:34:030
40812220,1sleep232-21ksoftirqd/210:42:152
2514399229,8cyclictest0-21swapper/110:35:201
25143992217,4cyclictest18226-21diskmemload11:33:311
25143992217,4cyclictest0-21swapper/112:15:171
25143992214,7cyclictest18226-21diskmemload09:13:271
2514999210,21cyclictest0-21swapper/210:16:092
2514999210,19cyclictest0-21swapper/211:30:152
25143992120,1cyclictest0-21swapper/109:36:001
25143992118,1cyclictest18226-21diskmemload09:15:151
2514099210,2cyclictest25888-21df_abs07:10:190
2514099210,18cyclictest0-21swapper/009:55:220
25153992018,1cyclictest361irq_work/311:20:403
25153992017,2cyclictest0-21swapper/311:19:283
25153992017,2cyclictest0-21swapper/311:19:283
2514099202,17cyclictest18226-21diskmemload12:26:360
2514099201,1cyclictest18226-21diskmemload09:53:220
2514099200,1cyclictest18226-21diskmemload11:47:120
2514099200,19cyclictest0-21swapper/011:35:180
2514099200,18cyclictest9731-21grep08:50:170
2514999192,15cyclictest0-21swapper/209:46:192
2514999191,17cyclictest20602-21ssh09:10:382
25143991915,1cyclictest320-21/usr/sbin/munin11:05:151
25143991911,7cyclictest17461-21hddtemp_smartct11:45:201
2514399190,18cyclictest0-21swapper/107:30:131
2514099193,15cyclictest11088-21open_files07:45:210
2514099191,18cyclictest18226-21diskmemload12:39:290
2514099191,14cyclictest5664-21gltestperf07:35:180
2514099191,14cyclictest5664-21gltestperf07:35:180
2514099191,14cyclictest18226-21diskmemload11:42:240
2514099190,18cyclictest0-21swapper/011:27:340
2514099190,15cyclictest0-21swapper/011:58:540
2515399180,17cyclictest0-21swapper/311:07:123
2514999183,1cyclictest31044-21ssh09:20:012
2514999181,3cyclictest141rcu_preempt10:21:302
2514999181,16cyclictest18226-21diskmemload10:53:142
2514999180,18cyclictest0-21swapper/211:17:432
2514999180,18cyclictest0-21swapper/211:17:432
2514399188,9cyclictest18226-21diskmemload09:53:021
2514399188,9cyclictest0-21swapper/112:14:241
25143991817,1cyclictest0-21swapper/110:49:291
25143991816,1cyclictest0-21swapper/112:00:161
25143991813,4cyclictest22903-21strings10:05:221
2514099183,14cyclictest32008-21sh09:20:160
2514099180,16cyclictest18226-21diskmemload12:32:120
25149991717,0cyclictest0-21swapper/211:35:182
2514999170,17cyclictest0-21swapper/211:26:272
2514999170,17cyclictest0-21swapper/210:59:142
2514399177,7cyclictest0-21swapper/110:26:441
2514399177,6cyclictest0-21swapper/111:16:511
2514399177,6cyclictest0-21swapper/111:16:501
2514399175,10cyclictest0-21swapper/110:02:221
25143991712,5cyclictest0-21swapper/112:25:381
25143991710,2cyclictest0-21swapper/111:50:161
2514099177,7cyclictest15657-21taskset09:02:510
2514099171,1cyclictest27774-21ntp_states08:20:210
2514099170,1cyclictest0-21swapper/011:21:560
2514099170,16cyclictest0-21swapper/008:55:290
2514999160,16cyclictest0-21swapper/209:26:012
2514999160,16cyclictest0-21swapper/209:26:012
2514399169,6cyclictest0-21swapper/109:21:331
25143991615,1cyclictest0-21swapper/111:35:431
25143991613,3cyclictest0-21swapper/111:41:121
25143991611,4cyclictest5663-21tail07:35:191
25143991611,4cyclictest5663-21tail07:35:181
2514099162,13cyclictest0-21swapper/009:14:210
2514099161,3cyclictest8993-21awk07:40:270
2515399156,6cyclictest3787-21df_abs12:00:173
25149991510,5cyclictest61750irq/125-enp2s0-09:38:132
2514999150,2cyclictest0-21swapper/212:29:452
2514999150,14cyclictest0-21swapper/212:13:172
2514999150,14cyclictest0-21swapper/211:02:042
2514399159,5cyclictest24345-21ssh10:33:041
2514399157,4cyclictest3250-21unixbench_multi08:35:271
2514099158,6cyclictest399-21tr08:30:210
2514099158,6cyclictest2815-21tr08:35:220
2514099155,6cyclictest23768-21taskset08:12:580
2514099151,1cyclictest6311-21munin-run08:45:000
25153991413,1cyclictest0-21swapper/308:45:143
2515399140,6cyclictest224-21systemd-journal12:26:303
2514999149,3cyclictest20178-21tail08:05:152
2514999148,2cyclictest1600-21sh11:05:262
25149991411,2cyclictest4421-21tr08:40:172
25149991410,3cyclictest345-21grep09:20:222
25149991410,2cyclictest4855-21cat07:35:002
2514999140,12cyclictest0-21swapper/209:40:132
2514399149,5cyclictest0-21swapper/110:22:451
2514399147,6cyclictest0-21swapper/107:10:191
2514399147,5cyclictest0-21swapper/109:05:301
2514399145,8cyclictest0-21swapper/110:54:421
2514399145,7cyclictest0-21swapper/111:58:541
2514399142,11cyclictest15914-21tr07:55:231
25143991410,3cyclictest0-21swapper/108:20:211
2514399140,1cyclictest0-21swapper/109:30:141
2514399140,14cyclictest0-21swapper/111:20:191
2514099148,5cyclictest16437-21seq07:55:400
2514099147,6cyclictest13673-21tr07:50:240
2514099141,9cyclictest32597-21apache_accesses07:25:120
2514099141,12cyclictest29275-21grep08:25:160
2514099140,1cyclictest0-21swapper/009:05:300
2514099140,1cyclictest0-21swapper/007:30:130
2514099140,12cyclictest0-21swapper/008:45:260
2515399139,3cyclictest28864-21grep11:55:003
2515399139,2cyclictest24933-21grep08:15:183
2515399139,2cyclictest11413-21basename09:30:153
2515399132,6cyclictest0-21swapper/307:30:133
25153991310,2cyclictest5458-21wc08:40:243
25153991310,2cyclictest5114-21rm09:50:573
25153991310,2cyclictest4646-21grep11:35:213
25153991310,2cyclictest29712-21cat11:55:163
25153991310,2cyclictest27420-21grep11:00:273
25153991310,2cyclictest14491-21grep09:00:173
2514999137,2cyclictest224-21systemd-journal12:20:002
25149991310,2cyclictest8989-21tail07:40:262
25149991310,2cyclictest8934-21ssh11:11:262
25149991310,2cyclictest31239-21apt-config08:30:002
25149991310,2cyclictest27680-21ssh10:10:012
25149991310,2cyclictest22421-21cron11:49:592
25149991310,2cyclictest16596-21df09:05:192
25149991310,2cyclictest12181-21wc07:50:002
2514999130,2cyclictest0-21swapper/211:50:142
2514399139,3cyclictest0-21swapper/112:35:161
2514399139,2cyclictest29186-21tr08:25:151
2514399137,6cyclictest0-21swapper/107:48:281
2514399137,4cyclictest29006-21apache210:13:291
2514399137,2cyclictest0-21swapper/110:57:131
25143991311,1cyclictest29263-21tr09:45:161
25143991310,2cyclictest24695-21grep08:15:161
25143991310,2cyclictest18226-21diskmemload12:07:351
2514399130,12cyclictest31769-21grep08:30:141
2514099138,4cyclictest28932-21ntpq07:15:230
2514099137,5cyclictest32222-21taskset07:23:410
2514099131,1cyclictest0-21swapper/008:15:160
2515399129,2cyclictest7027-21grep10:45:153
2515399129,2cyclictest32326-21cron07:25:003
2515399129,2cyclictest31174-21tune2fs11:30:183
2515399129,2cyclictest29362-21grep09:45:163
2515399129,2cyclictest28932-21tail10:10:173
2515399129,2cyclictest24393-21latency_hist09:15:003
2515399129,2cyclictest2323-21grep10:40:273
2515399129,2cyclictest22019-21sed10:30:273
2515399129,2cyclictest16726-21cut09:05:203
2515399129,2cyclictest15328-21sort07:55:193
2515399129,2cyclictest14872-21perf10:00:003
2515399129,2cyclictest13487-21cron10:25:003
2515399126,5cyclictest1378-21ntpd11:48:553
2515399122,7cyclictest19081-21/usr/sbin/munin10:55:203
25153991210,1cyclictest26845-21grep08:20:143
2515399120,7cyclictest224-21systemd-journal07:30:003
2515399120,1cyclictest0-21swapper/311:26:353
2514999129,3cyclictest0-21swapper/210:34:092
2514999129,2cyclictest9668-21grep08:50:172
2514999129,2cyclictest8676-21cron08:50:002
2514999129,2cyclictest6542-21grep07:35:262
2514999129,2cyclictest6542-21grep07:35:262
2514999129,2cyclictest30556-21wc12:20:242
2514999129,2cyclictest1313-21tr07:25:202
2514999127,2cyclictest11165-21chrt09:30:112
2514999126,5cyclictest188292chrt10:29:002
2514999126,5cyclictest15705-21anacron09:03:282
2514999120,4cyclictest0-21swapper/210:00:132
2514999120,1cyclictest0-21swapper/212:39:072
2514399129,2cyclictest9757-21cut09:55:181
2514399129,2cyclictest23605-21grep09:40:171
2514399128,3cyclictest30827-21sort07:20:161
2514399125,6cyclictest0-21swapper/111:10:181
25143991210,1cyclictest6329-21awk08:45:001
25143991210,1cyclictest18226-21diskmemload11:27:341
2514399120,11cyclictest9667-21grep08:50:171
2514099128,3cyclictest212862sleep008:06:370
2514099128,3cyclictest170352sleep008:00:090
2515399118,2cyclictest7075-21ssh11:10:183
2515399118,2cyclictest4995-21aten2_r7power_c09:25:153
2515399118,2cyclictest4995-21aten2_r7power_c09:25:143
2515399118,2cyclictest32307-21sort08:30:173
2515399118,2cyclictest32049-21ssh10:39:343
2515399118,2cyclictest31986-21df09:20:163
2515399118,2cyclictest30857-21ssh12:20:283
2515399118,2cyclictest28549-21tail07:15:183
2515399118,2cyclictest27519-21munin-setwarn07:15:003
2515399118,2cyclictest25022-21ssh09:40:383
2515399118,2cyclictest24015-21ssh10:05:503
2515399118,2cyclictest21990-21ssh12:15:023
2515399118,2cyclictest20795-21cut12:35:173
2515399118,2cyclictest20463-21seq09:37:373
2515399118,2cyclictest20185-21ssh10:03:503
2515399118,2cyclictest19829-21cat08:05:133
2515399118,2cyclictest15782-21ssh10:51:243
2515399118,2cyclictest15674-21apache_processe12:10:123
2515399118,2cyclictest14762-21grep10:25:233
2515399118,2cyclictest14401-21cron07:55:003
2515399118,2cyclictest13419-21sed12:30:173
2515399118,2cyclictest13281-21chrt08:58:093
2515399118,2cyclictest11223-21cat08:55:013
2515399117,3cyclictest1809-21cut10:15:143
2515399111,8cyclictest0-21swapper/307:40:143
2515399110,10cyclictest30074-21ssh09:19:073
2515399110,10cyclictest0-21swapper/311:40:133
2514999119,1cyclictest10956-21cut11:40:192
2514999118,2cyclictest29761-21sh10:37:062
2514999118,2cyclictest28090-21ssh10:10:122
2514999118,2cyclictest27020-21grep07:10:302
2514999118,2cyclictest26901-21tr08:20:152
2514999118,2cyclictest18722-21sed08:00:262
2514999118,2cyclictest14161-21chrt07:51:572
2514999118,2cyclictest12960-21sed08:55:292
2514999118,2cyclictest11661-21chrt12:05:492
2514999118,2cyclictest11330-21ssh10:48:512
2514999117,3cyclictest3290-21grep08:35:272
2514999110,11cyclictest0-21swapper/209:59:042
2514999110,11cyclictest0-21swapper/208:10:132
2514399118,2cyclictest8977-21grep07:40:261
2514399118,2cyclictest24122-21cat08:15:011
2514399118,2cyclictest13842-21expr07:50:261
2514399116,4cyclictest0-21swapper/109:25:171
2514399116,4cyclictest0-21swapper/109:25:171
2515399108,1cyclictest68702sleep307:37:553
2515399108,1cyclictest68702sleep307:37:553
2515399108,1cyclictest22029-21grep08:10:173
2515399107,2cyclictest9920-21apache_volume07:45:143
2515399107,2cyclictest9272-21ssh12:05:083
2515399107,2cyclictest29282-21tr08:25:163
2515399107,2cyclictest2352-21tail08:35:183
2514999108,1cyclictest28143-21expr07:15:162
2514999107,2cyclictest740-21grep08:30:252
2514999107,2cyclictest3336-21wget12:00:152
2514999107,2cyclictest31822-21ssh11:56:062
2514999107,2cyclictest30773-21df07:20:152
2514999107,2cyclictest25279-21cat08:15:202
2514999107,2cyclictest17574-21aten2_r7power_e11:20:142
2514999104,5cyclictest14681-21apt-get07:55:152
2514999100,9cyclictest14738-21ssh12:30:372
2514399108,1cyclictest29909-21taskset07:19:451
2514399108,1cyclictest20779-21cat08:05:201
2514399107,2cyclictest8353-21chrt08:45:531
2514399107,2cyclictest18196-21/usr/sbin/munin08:00:201
2514399107,2cyclictest11957-21cut08:55:201
2514399106,3cyclictest0-21swapper/107:25:141
251539997,1cyclictest18599-21cat08:00:253
251439997,1cyclictest13942-21tr09:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional