You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-01-31 - 05:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Celeron 33455 @1500 MHz, Linux 5.15.2-rt19 (Profile)

Latency plot of shadow in rack #c, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Tue Jan 31, 2023 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
170903724520,21sleep10-21swapper/119:05:151
170901822840,7sleep00-21swapper/019:05:130
181437821150,15sleep342-21ksoftirqd/300:05:243
1623991110,8rtkit-daemon1622-21rtkit-daemon22:15:263
1623991040,9rtkit-daemon1622-21rtkit-daemon21:27:563
1623991030,9rtkit-daemon1622-21rtkit-daemon21:21:363
1623991000,5rtkit-daemon0-21swapper/223:25:543
162399990,9rtkit-daemon1622-21rtkit-daemon20:13:053
17815192970,6sleep10-21swapper/122:30:281
162399960,8rtkit-daemon1622-21rtkit-daemon22:44:133
162399950,8rtkit-daemon1622-21rtkit-daemon22:56:123
162399950,4rtkit-daemon1803572-21apt23:35:180
17893032940,5sleep21789295-20cron22:55:012
17893032940,5sleep21789295-20cron22:55:002
18142692930,7sleep234-21ksoftirqd/200:05:192
171053629356,31sleep20-21swapper/219:09:522
162399890,8rtkit-daemon1622-21rtkit-daemon22:12:293
162399860,9rtkit-daemon1622-21rtkit-daemon19:35:203
162399850,8rtkit-daemon1622-21rtkit-daemon21:04:363
171025028468,9sleep30-21swapper/319:06:303
162399700,10rtkit-daemon1622-21rtkit-daemon20:18:403
1710703996819,44cyclictest1793217-21cat23:05:183
1710703996117,39cyclictest1795027-21cat23:10:223
162399590,9rtkit-daemon1622-21rtkit-daemon20:31:363
162399590,9rtkit-daemon1622-21rtkit-daemon19:50:023
17153682580,6sleep31715364-21sshd19:20:263
1710703995616,36cyclictest1791565-21expr23:00:193
162399560,9rtkit-daemon1622-21rtkit-daemon23:01:292
162399560,38rtkit-daemon0-21swapper/200:34:222
1710703995518,32cyclictest994-21lldpd19:34:383
1710703995514,37cyclictest1622-21rtkit-daemon22:37:413
162399540,9rtkit-daemon1622-21rtkit-daemon21:52:363
171070399532,45cyclictest1753-21gnome-shell21:30:103
1710703995318,31cyclictest1808301-21seq23:47:523
1710703995318,30cyclictest1779799-21cat22:25:303
1710703995318,30cyclictest1779799-21cat22:25:303
1710703995317,32cyclictest1823633-21cat00:30:383
171070399531,47cyclictest1753-21gnome-shell20:23:273
171070399522,44cyclictest1819767-21memory00:20:323
1710703995218,30cyclictest1770817-21expr22:00:203
162399520,8rtkit-daemon1622-21rtkit-daemon23:07:532
162399520,8rtkit-daemon1622-21rtkit-daemon21:23:122
162399520,8rtkit-daemon1622-21rtkit-daemon00:33:310
162399520,39rtkit-daemon906-21gmain20:30:422
171070399512,45cyclictest1753-21gnome-shell23:15:353
1710703995117,29cyclictest1761927-21cat21:35:193
1710703995115,31cyclictest994-21lldpd00:39:463
1710699995127,20cyclictest1753-21gnome-shell23:25:012
162399510,8rtkit-daemon1622-21rtkit-daemon23:55:020
162399510,8rtkit-daemon1622-21rtkit-daemon19:37:052
171070399503,43cyclictest1753-21gnome-shell23:20:363
171070399502,13cyclictest1753-21gnome-shell20:25:353
1710703995020,25cyclictest1753-21gnome-shell20:07:263
1710703995020,25cyclictest1753-21gnome-shell20:07:253
1710703995018,28cyclictest1803345-21cut23:35:013
1710703995014,13cyclictest1763897-21expr21:40:233
1710703995012,11cyclictest1742595-21cut20:40:023
18167242490,5chrt0-21swapper/000:10:420
171070399492,42cyclictest1753-21gnome-shell19:13:173
171070399492,42cyclictest1753-21gnome-shell00:19:523
1710703994922,23cyclictest1753-21gnome-shell19:27:273
1710703994919,26cyclictest1772218-21/usr/sbin/munin22:05:203
1710703994918,26cyclictest42-21ksoftirqd/321:05:223
1710703994917,28cyclictest1754777-21cut21:15:023
1710703994913,12cyclictest1777545-21ls22:20:183
171068699490,44cyclictest1622-21rtkit-daemon00:36:590
171070399482,42cyclictest1753-21gnome-shell20:02:493
171070399482,41cyclictest1753-21gnome-shell23:50:133
1710703994818,11cyclictest1722645-21sensors19:40:353
1710703994815,10cyclictest1788916-21uptime22:50:363
1710703994815,10cyclictest1788916-21uptime22:50:363
162399480,35rtkit-daemon1753-21gnome-shell19:30:181
171070399479,19cyclictest1753-21gnome-shell19:57:033
171070399472,41cyclictest1753-21gnome-shell23:44:433
171070399472,41cyclictest1753-21gnome-shell21:57:113
171070399472,41cyclictest1753-21gnome-shell20:47:303
1710703994721,23cyclictest1753-21gnome-shell23:39:163
1710703994721,22cyclictest1753-21gnome-shell20:50:343
1710703994715,11cyclictest42-21ksoftirqd/323:57:263
1710699994723,20cyclictest1753-21gnome-shell22:55:192
1710699994722,21cyclictest1753-21gnome-shell19:48:042
1710699994721,22cyclictest1753-21gnome-shell22:40:312
1710691994725,17cyclictest26-21ksoftirqd/119:15:021
171070399462,7cyclictest1753-21gnome-shell22:31:533
171070399462,7cyclictest1753-21gnome-shell19:54:113
1710703994618,23cyclictest1753-21gnome-shell20:43:243
1710703994616,26cyclictest42-21ksoftirqd/320:56:463
171070399461,40cyclictest1815714-21dpkg00:10:203
171070399461,39cyclictest1765326-21apt21:45:163
1710703994611,12cyclictest1812331-21cat00:00:203
1710703994611,11cyclictest1821333-21cat00:25:243
1710699994623,19cyclictest1753-21gnome-shell00:10:282
1710699994622,20cyclictest1753-21gnome-shell19:30:022
1710686994616,12cyclictest1069-21runrttasks21:49:450
162399460,31rtkit-daemon0-21swapper/119:25:521
171070399452,39cyclictest1753-21gnome-shell21:15:323
1710691994528,13cyclictest1777045-21sshd22:18:301
171070399449,31cyclictest1786983-21python322:45:333
1710703994419,21cyclictest1753-21gnome-shell19:15:073
1710703994419,21cyclictest1753-21gnome-shell19:15:073
1710699994422,19cyclictest1753-21gnome-shell22:11:572
1710699994420,20cyclictest1753-21gnome-shell20:53:292
1710699994416,11cyclictest438-20systemd-journal23:49:552
1710686994417,23cyclictest1742445-21unin-run20:40:010
162399440,31rtkit-daemon0-21swapper/223:27:032
171069999434,21cyclictest1782789-21cut22:35:172
1710699994320,19cyclictest1753-21gnome-shell20:55:192
1710699994318,20cyclictest1753-21gnome-shell21:05:222
17951432420,5sleep20-21swapper/223:10:222
17892912421,12sleep11789302-21grep22:55:001
17892912421,12sleep11789302-21grep22:55:001
17487672420,5chrt0-21swapper/120:55:351
1710691994222,16cyclictest26-21ksoftirqd/123:10:381
1710691994216,6cyclictest26-21ksoftirqd/121:40:261
171069199421,38cyclictest1753-21gnome-shell20:28:491
171069199421,15cyclictest1-21systemd20:20:011
1710686994223,14cyclictest1777829-21diskstats22:20:210
171068699421,37cyclictest1622-21rtkit-daemon00:21:400
162399420,30rtkit-daemon869-21irqbalance20:41:141
17375422410,5sleep2331rcuc/220:25:172
1710699994124,12cyclictest438-20systemd-journal22:01:552
171069999411,13cyclictest1753-21gnome-shell19:50:352
1710699994110,14cyclictest141rcu_preempt19:20:002
1710699994110,14cyclictest141rcu_preempt19:20:002
1710691994127,10cyclictest1742404-21cron20:39:591
1710691994126,12cyclictest1791042-21awk22:59:591
171068699417,12cyclictest141rcu_preempt19:20:010
171068699417,12cyclictest141rcu_preempt19:20:010
1710686994126,11cyclictest1737867-21taskset20:25:190
1710686994125,12cyclictest855-21dbus-daemon21:55:000
1710686994110,8cyclictest1753-21gnome-shell20:20:020
17734892400,8chrt1753-21gnome-shell22:07:300
1710699994011,6cyclictest141rcu_preempt21:00:202
1710691994025,10cyclictest1753398-21grep21:10:211
1710691994023,12cyclictest438-20systemd-journal23:25:291
1710691994023,12cyclictest1782879-21ls22:35:181
1710691994022,5cyclictest26-21ksoftirqd/119:54:591
171068699403,9cyclictest1753-21gnome-shell20:57:010
1710686994023,12cyclictest1793455-21expr23:05:200
171069999392,33cyclictest1753-21gnome-shell20:48:432
1710699993923,12cyclictest1810083-21sshd23:53:302
1710699993922,13cyclictest1812992-21ntpq00:00:332
1710699993921,13cyclictest855-21dbus-daemon20:20:012
1710699993920,14cyclictest2962348-21rs:main2
171069199392,33cyclictest1753-21gnome-shell22:07:001
1710691993922,11cyclictest438-20systemd-journal00:40:011
1710691993921,6cyclictest26-21ksoftirqd/120:20:181
1710691993914,12cyclictest26-21ksoftirqd/121:56:521
1710686993925,10cyclictest1754473-21sshd21:12:400
1710686993924,11cyclictest855-21dbus-daemon19:30:000
171068699392,33cyclictest1753-21gnome-shell23:34:000
171068699392,32cyclictest1753-21gnome-shell22:52:440
171068699392,32cyclictest1753-21gnome-shell22:52:440
1710686993922,12cyclictest438-20systemd-journal23:50:000
171068699392,18cyclictest1358-1kworker/u9:119:42:450
1710686993920,15cyclictest1752967-21unin-run21:10:010
1710686993920,14cyclictest438-20systemd-journal22:30:010
1710686993920,14cyclictest438-20systemd-journal22:30:010
18156252380,12sleep1251rcuc/100:10:021
17398482380,14sleep1251rcuc/120:30:271
171069999382,7cyclictest1753-21gnome-shell21:33:072
1710699993826,8cyclictest1753-21gnome-shell00:16:402
171069999382,32cyclictest1753-21gnome-shell21:12:482
1710699993822,12cyclictest1821207-21expr00:25:202
1710699993820,13cyclictest438-20systemd-journal21:17:012
1710699993811,5cyclictest141rcu_preempt19:30:192
171069999381,12cyclictest1753-21gnome-shell23:31:172
1710691993826,8cyclictest1069-21runrttasks23:20:341
171069199382,32cyclictest1753-21gnome-shell00:00:131
1710691993822,12cyclictest1-21systemd00:00:001
1710691993820,6cyclictest26-21ksoftirqd/122:10:361
1710691993820,13cyclictest1722724-21cat19:40:371
1710691993818,15cyclictest1805417-21cat23:40:221
171069199381,32cyclictest1753-21gnome-shell23:31:081
171068699387,15cyclictest13-21ksoftirqd/021:28:500
1710686993825,10cyclictest1810229-21awk23:55:000
1710686993823,11cyclictest855-21dbus-daemon21:35:010
1710686993823,11cyclictest1756389-21sh21:19:590
162399380,25rtkit-daemon0-21swapper/119:18:421
162399380,25rtkit-daemon0-21swapper/119:18:421
162399380,23rtkit-daemon856-21gmain00:15:581
17302992370,9sleep01753-21gnome-shell20:05:010
1710699993726,7cyclictest1753-21gnome-shell23:38:312
171069999372,31cyclictest1753-21gnome-shell21:27:102
1710699993713,20cyclictest1753-21gnome-shell21:50:012
171069199374,12cyclictest26-21ksoftirqd/121:37:441
1710691993724,8cyclictest1753-21gnome-shell22:20:201
1710691993724,10cyclictest1771984-21awk22:05:001
1710691993722,12cyclictest1768524-21awk21:55:011
1710691993721,12cyclictest1720317-21expr19:35:191
1710691993721,12cyclictest1715564-21ntpq19:20:281
1710691993720,13cyclictest26-21ksoftirqd/121:00:191
171068699372,31cyclictest1764000-21hddtemp_smartct21:40:290
1710686993722,11cyclictest1769817-21sed21:55:350
1710686993722,11cyclictest1736853-21uptime20:20:440
171068699372,14cyclictest1358-1kworker/u9:123:15:590
1710686993721,12cyclictest1812840-21memory00:00:300
1710686993720,13cyclictest1745096-21grep20:45:280
1710699993624,8cyclictest1753-21gnome-shell19:10:322
171069999362,30cyclictest1753-21gnome-shell19:40:322
171069999362,30cyclictest0-21swapper/220:05:332
171069999362,30cyclictest0-21swapper/220:05:332
1710699993622,10cyclictest34-21ksoftirqd/220:14:562
171069999361,9cyclictest0-21swapper/222:47:492
171069999361,15cyclictest141rcu_preempt22:05:202
171069199366,11cyclictest26-21ksoftirqd/121:07:091
1710691993624,7cyclictest26-21ksoftirqd/120:10:351
1710691993620,12cyclictest1816331-21sed00:10:321
1710691993620,11cyclictest1809532-21awk23:50:331
171069199361,9cyclictest0-21swapper/123:16:391
1710691993619,12cyclictest438-20systemd-journal23:05:291
1710691993615,6cyclictest26-21ksoftirqd/121:32:211
1710691993610,6cyclictest141rcu_preempt23:00:201
171068699368,15cyclictest141rcu_preempt19:21:430
171068699362,6cyclictest0-21swapper/023:21:480
1710686993624,9cyclictest1800985-21swap23:25:340
171068699362,21cyclictest0-21swapper/020:31:340
171068699362,17cyclictest1814782-21ps00:05:340
1710686993620,12cyclictest1821206-21expr00:25:190
1710686993620,11cyclictest1726949-21cat19:55:140
1710686993619,13cyclictest1762315-21cat21:35:250
171068699361,21cyclictest1774796-21ps22:10:350
171069999359,7cyclictest1825833-21cron00:40:002
1710699993518,12cyclictest438-20systemd-journal21:51:292
1710699993518,12cyclictest1729744-21grep20:00:352
171069999351,6cyclictest141rcu_preempt23:40:252
171069999351,31cyclictest1743943-21sshd20:42:352
1710691993524,6cyclictest26-21ksoftirqd/120:45:211
1710691993522,9cyclictest1821962-21sort00:25:391
1710691993522,7cyclictest26-21ksoftirqd/100:30:191
1710691993519,12cyclictest1766361-21sed21:45:361
1710691993518,6cyclictest26-21ksoftirqd/121:19:591
1710691993518,13cyclictest1753-21gnome-shell22:46:231
1710691993517,12cyclictest438-20systemd-journal20:50:031
171068699353,12cyclictest1358-1kworker/u9:123:04:160
171068699352,7cyclictest1726700-21grep19:55:000
171068699352,30cyclictest0-21swapper/020:13:340
171068699352,17cyclictest1818041-21memory00:15:380
1710686993520,11cyclictest918-21gdbus21:01:570
1710686993520,11cyclictest1746562-21basename20:50:240
1710686993520,11cyclictest1720700-21tr19:35:260
1710686993519,12cyclictest1796079-21sshd23:11:300
1710686993519,12cyclictest1743691-21cat20:40:400
1710686993519,12cyclictest1718620-21expr19:30:190
1710686993519,11cyclictest1782185-21anacron22:32:490
1710686993518,12cyclictest1775819-21sort22:15:190
1710686993516,14cyclictest1785143-21ntpq22:40:360
171068699351,15cyclictest1358-1kworker/u9:122:03:400
162399350,5rtkit-daemon1753-21gnome-shell23:42:100
1710699993423,7cyclictest1714848-21sh19:20:162
171069999342,12cyclictest0-21swapper/200:20:202
1710699993420,10cyclictest34-21ksoftirqd/222:25:182
1710699993420,10cyclictest34-21ksoftirqd/222:25:182
1710699993418,12cyclictest1781930-21sed22:30:372
1710699993417,13cyclictest1753-21gnome-shell20:35:312
1710699993417,12cyclictest438-20systemd-journal20:20:292
1710691993422,7cyclictest26-21ksoftirqd/122:25:341
1710691993422,7cyclictest26-21ksoftirqd/122:25:341
1710691993421,8cyclictest26-21ksoftirqd/100:20:401
1710691993419,11cyclictest1784919-21cat22:40:321
1710691993419,11cyclictest1727310-21expr19:55:191
1710691993418,12cyclictest855-21dbus-daemon23:40:021
1710691993417,13cyclictest1730818-21fw_packets20:05:231
1710691993417,13cyclictest1730818-21fw_packets20:05:231
1710691993414,5cyclictest141rcu_preempt21:26:291
171069199341,29cyclictest855-21dbus-daemon21:25:011
171068699342,19cyclictest1826-21JS14:31:440
171068699342,12cyclictest0-21swapper/022:35:140
1710686993419,11cyclictest1786502-21fw_packets22:45:240
1710686993418,12cyclictest1757015-21expr21:20:190
1710686993418,11cyclictest1731665-21sshd20:07:270
1710686993418,11cyclictest1731665-21sshd20:07:260
171068699341,29cyclictest0-21swapper/022:55:140
171068699341,12cyclictest141rcu_preempt19:10:270
17647232334,5sleep2141rcu_preempt21:41:012
1710699993322,7cyclictest1769458-21grep21:55:262
1710699993318,11cyclictest1797259-21wc23:15:272
1710699993315,14cyclictest2118-21evolution-alarm23:59:592
1710699993315,13cyclictest2962348-21rs:main2
1710691993323,6cyclictest26-21ksoftirqd/123:50:001
1710699993217,11cyclictest1762395-21cat21:35:272
1710699993216,11cyclictest1776138-21idleruntime22:15:272
1710691993221,6cyclictest26-21ksoftirqd/119:45:341
1710691993217,11cyclictest1728871-21grep20:00:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional